Academic Research

LeNet-5 is a pioneering convolutional neural network developed by Yann LeCun and colleagues in 1998. It was designed for handwritten digit recognition and is considered one of the foundational architectures in deep learning, establishing many patterns still used in modern CNNs.

AlexNet is a landmark convolutional neural network that won the ImageNet Large Scale Visual Recognition Challenge in 2012 by a significant margin. Developed by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, it sparked the deep learning revolution in computer vision.

VGG is a deep convolutional neural network architecture developed by the Visual Geometry Group at Oxford. Known for its simplicity and depth (16-19 layers), VGG demonstrated that network depth is critical for good performance and became widely used for transfer learning.

ViLT (Vision-and-Language Transformer) is a minimal vision-and-language model that processes raw image patches directly without using a separate visual encoder like CNNs or region features. This makes it significantly faster while maintaining competitive performance.

ERNIE (Enhanced Representation through kNowledge IntEgration) is a series of language models developed by Baidu. It incorporates knowledge graphs and entity-level masking to achieve better understanding of semantic relationships and world knowledge.

GPT-J is a 6 billion parameter open-source autoregressive language model developed by EleutherAI. It was one of the first large-scale open alternatives to GPT-3 and demonstrated that the open-source community could train competitive language models.