Course curriculum

  • 1

    Introduction

    • Course Introduction

    • Course Overview

    • Neural Network Defined

    • Framework for Optimal Learning

    • Optimal Learning Techniques

    • Optimal Generalizations Techniques

    • Optimal Prediction Techniques

    • Framework Application

    • Diagnostic Learning Curves

    • The Fit of the Model

    • Unrepresentative Dataset
  • 2

    Optimal Learning

    • Neural Networks Learn a Mapping Function

    • Error Surface

    • Features of the Error Surface

    • Non-Convex Error Surface

    • Deep Learning Neural Network Components: Part 1

    • Deep Learning Neural Network Components: Part 2

    • Neural Network Model Capacity

    • Anatomy of a Keras Model

    • Demo: Case Study on Model Capacity: Part 1

    • Demo: Case Study on Model Capacity: Part 2

    • Demo: Case Study on Model Capacity: Part 3

    • Gradient Precision with Batch Size

    • Demo: Case Study on Batch Size: Part 1

    • Demo: Case Study on Batch Size: Part 2

    • Demo: Case Study on Batch Size: Part 3

    • Loss Function Defined

    • Choosing a Loss Function

    • Demo: Case Study on Regression Loss Functions: Part 1

    • Demo: Case Study on Regression Loss Functions: Part 2

    • Demo: Case Study on Binary Classification Loss Functions: Part 1

    • Demo: Case Study on Binary Classification Loss Functions: Part 2

    • Demo: Case Study on Binary Classification Loss Functions: Part 3

    • Demo: Case Study on Multiclass Classification Loss Functions: Part 1

    • Demo: Case Study on Multiclass Classification Loss Functions: Part 2

    • Learning Rate Defined

    • Configuring the Learning Rate

    • Learning Rate Schedules and Adaptive Learning Rates

    • Defining Learning Rates in Keras

    • Demo: Case Study on Learning Rates: Part 1

    • Demo: Case Study on Learning Rates: Part 2

    • Demo: Case Study on Learning Rates: Part 3

    • Demo: Case Study on Learning Rates: Part 4

    • Data Scaling

    • Scaling the Input and Ouput Variables

    • Normalize and Standardize (Rescaling)

    • Demo: Case Study on Data Scaling: Part 1

    • Demo: Case Study on Data Scaling: Part 2

    • Demo: Case Study on Data Scaling: Part 3

    • Demo: Case Study on Data Scaling: Part 4

    • Activation Functions and Vanishing Gradients

    • Rectified Linear Activation Function Defined and Implemented in Python

    • Rectified Linear Activation Function Defined and Implemented in Python

    • When ReLU is the Appropriate Choice

    • Demo: Case Study on Vanishing Gradients: Part 1

    • Demo: Case Study on Vanishing Gradients: Part 2

    • Correct Exploding Gradients with Clipping

    • Gradient Clipping in Keras

    • Demo: Case Study on Exploding Gradients Part 1

    • Demo: Case Study on Exploding Gradients Part 2

    • Batch Normalization

    • Tips for Applying Batch Normalization

    • Demo: Case Study on Batch Normalization: Part 1

    • Demo: Case Study on Batch Normalization: Part 2

    • Greedy Layer-Wise Pretraining

    • Demo: Greedy Layer-Wise Pretraining Case Study
  • 3

    Optimal Generalization

    • The Problem of Overfitting

    • Reduce Overfitting by Constraining Complexity

    • Regularization Approaches for Neural Networks

    • Penalize Large Weights via Regularization

    • How to Penalize Large Weights

    • Tips for Using Weight Regularization

    • Demo: Weight Regularization Case Study: Part 1

    • Demo: Weight Regularization Case Study: Part 2

    • Activity Regularization

    • Encouraging Smaller Activations

    • Tips for Activity Regularization

    • Activity Regularization in Keras

    • Demo: Activity Regularization Case Study

    • Forcing Small Weights

    • How to Use a Weight Constraint

    • Tips for Appling Weight Constraints

    • Weight Constraints in Keras

    • Demo: Weight Constraint Case Study

    • Dropout

    • Dropout Mechanics

    • Dropout Tips

    • Dropout in Keras

    • Demo: Dropout Case Study

    • Noise Regularization

    • How to add Noise

    • Noise Tips

    • Adding Noise in Keras

    • Demo: Noise Regularization Case Study
  • 4

    Optimal Predictions

    • Ensemble Learning

    • Ensemble Neural Network Models

    • Varying the Major Elements

    • Model Averaging Ensembles

    • Ensembles in Keras

    • Demo: Model Averaging Ensemble Case Study: Part 1

    • Demo: Model Averaging Ensemble Case Study: Part 2

    • Demo: Model Averaging Ensemble Case Study: Part 3

    • Weighted Average Ensembles

    • Demo: Weighted Average Ensemble Case Study: Part 1

    • Demo: Weighted Average Ensemble Case Study: Part 2

    • Demo: Weighted Average Ensemble Case Study: Part 3

    • Demo: Weighted Average Ensemble Case Study: Part 4

    • Resampling Ensembles

    • Demo: Resampling Ensemble Case Study: Part 1

    • Demo: Resampling Ensemble Case Study: Part 2

    • Demo: Resampling Ensemble Case Study: Part 3

    • Demo: Resampling Ensemble Case Study: Part 4

    • Horizontal Voting Ensembles

    • Demo: Horizontal Ensemble Case Study: Part 1

    • Demo: Horizontal Ensemble Case Study: Part 2