Posts tagged "engineering"

Backpropagation is simple in theory and unforgiving in practice. Here's what actually went wrong when I built one with nothing but numpy, and how gradient checking saved me.

The part of the job that's shrinking isn't the part most people worry about. It's not thinking that's getting outsourced — it's typing.

Pipelines rarely fail loudly. They fail by quietly producing plausible-looking wrong answers, which is a much harder problem than a crash.

Stripped of the vector calculus notation, gradient descent is just a rule for guessing better. Here's the mechanical version I wish someone had shown me first.