Basic Techniques
- AlexNet: Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. “Imagenet classification with deep convolutional neural networks.” Advances in neural information processing systems. 2012.
- VGGNet: Simonyan, Karen, and Andrew Zisserman. “Very deep convolutional networks for large-scale image recognition.” arXiv preprint arXiv:1409.1556 (2014).
- ResNet: He, Kaiming, et al. “Deep residual learning for image recognition.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
- GoogLeNet: Szegedy, Christian, et al. “Going deeper with convolutions.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.
- Batch Normalization: Ioffe, Sergey, and Christian Szegedy. “Batch normalization: Accelerating deep network training by reducing internal covariate shift.” arXiv preprint arXiv:1502.03167 (2015).
- Inception v3: Szegedy, Christian, et al. “Rethinking the inception architecture for computer vision.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
- Inception v4: Szegedy, Christian, et al. “Inception-v4, inception-resnet and the impact of residual connections on learning.” Thirty-First AAAI Conference on Artificial Intelligence. 2017.
- DenseNet: Huang, Gao, et al. “Densely connected convolutional networks.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
- MobileNet v1: Howard, Andrew G., et al. “Mobilenets: Efficient convolutional neural networks for mobile vision applications.” arXiv preprint arXiv:1704.04861 (2017).
- MobileNet v2: Sandler, Mark, et al. “Mobilenetv2: Inverted residuals and linear bottlenecks.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
- ShuffleNet v1: Zhang, Xiangyu, et al. “Shufflenet: An extremely efficient convolutional neural network for mobile devices.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
- ShuffleNet v2: Ma, Ningning, et al. “Shufflenet v2: Practical guidelines for efficient cnn architecture design.” Proceedings of the European Conference on Computer Vision (ECCV). 2018.
Loss Function of Face Detection
- CenterLoss: Wen, Yandong, et al. “A discriminative feature learning approach for deep face recognition.” European conference on computer vision. Springer, Cham, 2016.
- SphereFace: Liu, Weiyang, et al. “Sphereface: Deep hypersphere embedding for face recognition.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
- AMsoftmax: Wang, Feng, et al. “Additive margin softmax for face verification.” IEEE Signal Processing Letters 25.7 (2018): 926-930.
- ArcFace: Deng, Jiankang, et al. “Arcface: Additive angular margin loss for deep face recognition.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.
Attention Models
- SENet: Hu, Jie, Li Shen, and Gang Sun. “Squeeze-and-excitation networks.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
- Non-local Neural Network: Wang, Xiaolong, et al. “Non-local neural networks.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
- GCNet: Cao, Yue, et al. “GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond.” arXiv preprint arXiv:1904.11492 (2019).
- SGENet: Li, Xiang, Xiaolin Hu, and Jian Yang. “Spatial Group-wise Enhance: Improving Semantic Feature Learning in Convolutional Networks.” arXiv preprint arXiv:1905.09646 (2019).
Neural Architecture Search
- MobileNet v3: Howard, Andrew, et al. “Searching for mobilenetv3.” arXiv preprint arXiv:1905.02244 (2019).
- FBNet: Wu, Bichen, et al. “Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019.
Model Parallelism
- CNN Parallelism: Krizhevsky, Alex. “One weird trick for parallelizing convolutional neural networks.” arXiv preprint arXiv:1404.5997 (2014).
Natural Language Processing
- Transformer: Vaswani, Ashish, et al. “Attention is all you need.” Advances in neural information processing systems. 2017.
- BERT: Devlin, Jacob, et al. “Bert: Pre-training of deep bidirectional transformers for language understanding.” arXiv preprint arXiv:1810.04805 (2018).
Overview
- Object Detection: Wu, Xiongwei, Doyen Sahoo, and Steven CH Hoi. “Recent Advances in Deep Learning for Object Detection.” arXiv preprint arXiv:1908.03673 (2019).
- Generative Adversarial Network: Creswell, Antonia, et al. “Generative adversarial networks: An overview.” IEEE Signal Processing Magazine 35.1 (2018): 53-65.
- Sentiment Analysis: Zhang, Lei, Shuai Wang, and Bing Liu. “Deep learning for sentiment analysis: A survey.” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 8.4 (2018): e1253.
- Gradient Descent Optimization: Ruder, Sebastian. “An overview of gradient descent optimization algorithms.” arXiv preprint arXiv:1609.04747 (2016).