I know, okay?

I took an official break for 4th of July weekend, when I went to Austin and stood in line for barbecue for hours for the first time ever and shot a bunch of guns for the first time ever ("No shooting at propane tanks, no redneck stuff," warned the guy at the counter), and then I took an unofficial break when I got back from Wednesday through Saturday, because reasons. On at least one of those days, I may or may not have found out that a girl I liked was seeing someone and stayed up all night pretending I was a spaceship captain because, you know, you can outrun all your emotional problems if you jump to hyperspace quickly enough.

Yesterday though, I was at a party where I actually got to explain neural networks and use the phrase "universal approximator" and people were polite enough to not excuse themselves immediately in a wild-eyed panic and even asked if they could read this blog. What kind of parties am I going to? The kind where I also get to learn to drink vodka "The Russian Way" (first you expel the air from your lungs, then you drink the vodka, and then you bite into a pickle), get into paper plate fights with guests, and recruit people for a flashmob at a Jewish wedding. I had a lot of fun is I guess what I'm trying to say.

Xavier Initialization

Remember how in the spreadsheet neural network we built on day eleven, the final activations our network generated were just all over the place? Our desired outputs were 0 and 1, and we were getting guesses in the hundreds and thousands.

Our day twelve Keras model did a much better job - not just because it included a optimization process that got us to a much better answer after a few runs, but because the Keras model had built-in weight initialization methods that gave us something a lot closer to what we wanted on its very first guess.

Let's go back to our spreadsheet to check out how weight initialization might work.