All notes

19 notes, newest first

All posts

#019
Why Learning Rates Should Decay During Training
Machine Learning8 min
#018
Adam From First Principles
Machine Learning4 min
#017
RMSProp: Why Every Parameter Shouldn't Take the Same Step
Machine Learning4 min
#016
Why Gradient Descent Needs Momentum
Machine Learning4 min
#015
Why Mini-Batch Gradient Descent Works
Machine Learning5 min
#014
Why Deep Neural Networks Become Hard to Train: Vanishing Gradients, Exploding Gradients, and Weight Initialization
Machine Learning12 min
#013
Understanding Regularization Through Gradient Descent
Machine Learning12 min
#012
The Tax You Can't Avoid Paying
Learning6 min
#011
Information Is Not Knowledge
Philosophy5 min
#010
Lessons From Implementing a Neural Network From Scratch
Machine Learning6 min