Siraj Raval's How to Do Mathematics Easily links to three cheat sheets that promise to cover the entirety of mathematics required to understand the basics of neural networks:

• Linear algebra
• Calculus
• Statistics

But cheat sheets only work if you have some kind of handle on the underlying material, and I've gone my entire adult life without an understanding of how derivatives work. Sometimes I think it's just the transformation of a mathematical term into some other term, then someone throws a graph in there and I'm in the woods again.

Deep breaths. Here goes nothing.

Ten minutes later

Is it possible the answer to the question "what is a derivative" has been "the rate of change of a thing" THIS WHOLE TIME AND I NEVER FIGURED THAT OUT?

It's all so simple. I'm out here plotting functions and calculating tangents. I feel like I'm this close to a mathematical understanding of gradient descent.

Leave me alone. It's Saturday night, okay?