Skip to main content

Classification Algorithms


Links to important articles for quick revision:

Residual Netwoks- ResNet vs ResNeXt , ResNet Variants

  • ResNet - Vanishing gradient problem is addressed for deeper networks. Identity shortcut connections and pre-activation residual connection are best as per ablation studies  
  • ResNeXt- Cardinality introduced, 
    • split transform aggregate strategy (Layers with 64 width are broken down into 32 cardinality with 4 width), kind of ensemble technique. 
    • Number of FLOPS and Parameters remain same for ResNet50 and ResNeXt-50
    • Width vs Cardinality - More cardinality preferred over width 
    • ResNeXt-32*4d and ResNeXt-64*4d are mostly used in pytorch
  • DenseNet - Skip connection from one layer to all other layers 

Inception: Inception Blog

  • Inception V1 - Multiple sized kernels for convolution (More width than depth)
  • Inception V2 - Factorized Convolutions (3x3 rather than 5x5, nx1 and 1xn rather than nxn), filter banks in the module were expanded 
  • Inception v3 - [2015 CVPR] RMSProp optimizer, factorized 7x7 convolutions, batchnorm in auxillary classifier, label smoothing
  • Inception v4 - Stem changed to inception blocks, explicit reduction blocks introduced, more uniform Inception A, B, C blocks
  • Inception Resnet (v1 and v2) - Skip connections were added to each inception block, InceptionResnet-v1 similar to v3 and InceptionResnet-v2 similar to v4 in terms of computational cost
  • Xception - Xception Blog [2017 CVPR]
    • Extreme Version of Inception
    • Reversed Depthwise separable convolution layers used (Pointwise + Depthwise Convolution)
    • Beats Inception v3 in ImageNet challenge accuracy

MobileNet: MobileNet Blog [2018 CVPR]

  • MobileNet v1 - Computationally efficient, Depthwise Separable Convolution , width and resolution scaling multipliers 
  • MobileNet v2 - Inverted residual structure, linear bottlenecks, shortcut between bottlenecks
  • MobileNetv3 [ICCV 2019] Explanation

EfficientNet: Medium Blog Blog2 [2019 ICML]

  • Efficent Net [B0-B7] - Scale depth, width and resolution as per constrained rule 



Comments

Post a Comment

Popular posts from this blog

Most Important steps to win any Machine Learning Competition

Gaining experience from the blogs of all the winners of online competition, I could see what approaches they follow, their solutions etc. Some of the most important points that I try to follow when I see a new machine learning challenge are: 1. Understand : Try to understand the data thoroughly, what each feature means and how it it related to the target variable. Exploring the data using plots gives an idea about features.  2. Data cleaning : Imputation of missing values and data pre-processing.  3. Feature Engineering : The most important part is feature engineering. Good feature engineering can give you an edge over others in any competition. In fact, it's the most important and most difficult step to think of and comes only with experience and practice. 4. Splitting your data:  Creating a strategy how to split your data into training and validation set. A good validation set that matches your score on the leader board is very crucial to fine tune your code....
Post a Comment
Read more

Interesting Projects in Deep Learning

Deep Learning is the latest new area in machine learning which is proving handy to solve many interesting problems which were not possible to do with the standard ML algorithms before such as Language Translation, Text to speech, Speech Recognition etc. It has moved us one step closer to what we thought of as our objective. Self Driving cars are the latest invention of this field and is growing at a very fast rate with more number of developers opting to move to deep learning. Some of the most interesting projects in deep learning that helps you learn concepts as well as apply these concepts on real world data and complex use cases are: Tranfer Learning : To use a pretrained model to find objects in any image Style Transfer : Applying the style of one image onto the other image Digit Recognizer : Use CNN to find the handwritten digits with our own custom built CNN model MNIST data : Classify over 10 objects in the MNIST data TED Talks : Creating word embeddings in a dataset...
Post a Comment
Read more