Deep Learning

This site is dedicated to the simplest video tutorials on Deep Learning. My aim is to prepare a free & interactive video book on Deep Learning. I have used my knowledge and experience to prepare these tutorials. All feedback and suggestions are welcome (email me at nirajrkumar@gmail.com or nirajrkumar@yahoo.com).

As, scientific development is an endless process, so I will keep updating it. Clicking on the link will drive you to the YouTube page for related content. Or You can use the link: https://www.youtube.com/c/DrNirajRKumar

  1. Bias Variance Tradeoffs.

Content:

  • Basics of Bias and Variance
  • Bias and variance using bulls-eye diagram.
  • Bias Variance Tradeoff and Algorithm Design Strategies

Video Link:

  1. Gradient Descent Algorithms.

Content:

  • Gradient Descent
  • Batch Gradient Descent
  • Stochastic Gradient Descent
  • Mini Batch Gradient Descent

Video Link:

  1. Vanishing and Exploding Gradient Problems.

Content:

  • Key Reasons for Vanishing and Exploding Gradient
  • Symptoms to Identify
  • How to Solve the problem of Vanishing and Exploding Gradients (some key approaches)
  • Weight Initialization
  • Gradient Clipping
  • Batch Normalizations

Video Link:

Vanishing and Exploding Gradient Problems.

  1. Momentum, RMSProp, and ADAM Optimizers.

Content:

  • Momentum Optimizers
  • RMSProp Optimizers
  • ADAM Optimizers

Video Link:

  1. Deep Learning Cost Functions.

Content:

  • Entropy and Cross-Entropy
  • Binary Cross Entropy Loss
  • Categorical Cross Entropy Loss

Video Link:

  1. Regularization in Deep Learning.

Content:

  • L1-Regularization
  • L2-Regularization
  • L1 Regularization in Deep Learning and Sparsity
  • L2 Regularization in Deep Learning and Weight Decay

Video Link:

  1. Saddle Points Problem in Deep Learning.

Content:

  • Saddle Points - Introduction
  • Impact of Saddle Points in Learning
  • Second Partial Derivative Test
  • Eigen Value & Hessian Matrix based Test.
  • How to Escape from Saddle points

Video Link:

  1. CNN (Convolutional Neural Networks).

Content:

  • Part-1: Basics of CNN,
  • Convolution Operation,
  • Pooling Operation, and Calculating I/O size,
  • Part-2: Basic of Padding in Convolutional Neural Network,
  • The Padding I/O Size Computation,
  • Some Insight about Padding,
  • Part-3: Basics of Stride in Convolutional Neural Network,
  • Convolution and Pooling Operation with Higher Stride Values.,
  • Some Insight about Strides

Video Link:

  1. Deep Learning using Deep Neural Network.

Content:

  • Basics of Deep Learning and Deep Neural Networks,
  • Forward Propagation and Total Error Computation
  • Back Propagation in Deep Neural Networks

Video Link:

  1. RNN (Recurrent Neural Network).

Content:

Part-1
  • Basics of Recurrent Neural Network (RNN),
  • Architectural differences between RNN and DNN (Deep Neural Network).
  • Training RNN - Through a very simple example.
Part-2
  • Explains the use of "Back-propagation Through Time (BPTT)", to train the RNN.

Video Link:

  • RNN - Part-1 (Direct Link: https://www.youtube.com/watch?v=SiBLzJxSzpo)

  • RNN Part-2 (Direct Link: https://www.youtube.com/watch?v=_ugj7u96_Zk)

  1. LSTM (Long Short Term Memory).

Content:

Part-1
  • Basics of LSTM (Long Short Term Memory),
  • Vanishing Gradient Problem in RNN
  • Architectural differences between RNN and LSTM.
  • Feed Forward Propagation
Part-2
  • Explains the Forward Propagation and "Back-propagation Through Time (BPTT)", to train the LSTM.
Part-3
  • Derivation for the "Back-propagation Through Time (BPTT)", to train the LSTM.

Video Link:

  1. Restricted Boltzmann Machine (RBM).

Content:

Part-1
  • Basic Overview of RBM and
  • Applications of RBM
Part-2
  • Constructive divergence,
  • Gibbs Sampling,
  • Using Gibbs Sampling for training.
Part-3
  • Continue Training process for RBM.

Video Link:

  1. Deep Learning using Deep Belief Network.

Content:

  • Part-1
  • Deep Belief Network Basics and
  • Working of the DBN Greedy Training through an example.
  • Part-2:
  • Working of the DBN Greedy Training (continue from part-1)
  • Fine tuning of Deep Belief Networks,
  • Classification using Deep Belief Networks

Video Link:

  1. Attention-Based Models.

Content:

  • Self-Attention
  • Hierarchical Attention

Video Link: