In this course, you will learn how to build your own neural network optimizers so that you can train your neural networks on data. This course is Part 3 in the Flamethrower Core series.You'll learn about various concepts associated with neural network optimization such as loss functions, gradient descent, learning rates, and hyperparameter search. You'll also learn about the mathematical and theoretical underpinnings of these strategies such as Maximum Likelihood Estimation and Maximum A Posteriori. As each of these concepts is introduced, you'll implement them in your library and test them out on real world data. View the entire course syllabus below, along with preview lessons. Be sure to click the drop down arrow to see the syllabus in its entirety.
Course Curriculum
-
An Introduction to Neural Network Optimization
(21:04)
-
The Optimizer Base Class
(13:08)
-
Loss Functions - Introduction
(7:37)
-
Loss Functions - Maximum Likelihood Estimation, KL Divergence, Wasserstein Metrics
(14:09)
-
Loss Functions - Deriving Mean Squared Error
(17:05)
-
Loss Functions - Deriving Cross Entropy Loss
(13:24)
-
Loss Functions - Maximum A Posteriori
(13:24)
-
Loss Functions - Maximum a Posteriori with a Gaussian Prior
(7:38)
-
Loss Functions - Maximum a Posteriori with a Laplacian Prior
(9:14)
-
Loss Functions - The Implementation
(14:35)
-
Gradient Descent - The Theory
(41:01)
-
Improving Gradient Descent - Momentum
(13:19)
-
Improving Gradient Descent - Nesterov Momentum
(5:58)
-
Gradient Descent - The Implementation
(11:15)
-
Scheduling Learning Rates
(33:34)
-
Hyperparameter Search
(12:53)