math
-
Introduction In the previous blog post we took a look at the backpropagation through time algorithm. We saw how, looking at the unfolded computation graph, backpropagation through time is essentially the same as backpropagating through one long connected feedforward neural net. Now that we have the gradient of the loss with respect to the parameters…
-
Introduction In the previous blog post, we began the implementation of our feedforward neural net, and went over the equations and code for the forward pass. With this, we can now feed our data through the network, but its not of much use without being able to train it. In order to train our recurrent…
-
Introduction In the previous blog post we talked about what it means exactly for data to have a “sequential topology”, and prepared our training data to be fed into the recurrent neural net. In this post, we’ll be looking at the design of our simple recurrent neural net, the equations for the forward pass, and…
-
Introduction In the previous blog post we looked at how a recurrent neural network differs from a feedforward neural network, and why they are better at sequence processing tasks. In this blog post, we will be preparing our training data, and start writing some code. Our training data is just a list of lowercase names.…
-
Introduction In a previous series of blog posts I covered feedforward neural networks. Feedforward neural networks are very powerful, but they are not the only neural network architecture available and they may be ill-suited to certain tasks. Recurrent neural nets are a class of neural nets that are particularly effective for modeling data with a…
-
Introduction This is the third installment in a series of blog posts about Support Vector Machines. If you have not read the first two blog posts, I highly recommend you go back and read those before continuing to this blog post. Last time we looked at some of the theory behind constrained optimization, and reformulated…
-
Every matrix represents a linear transformation. A special kind of matrix, called a diagonal matrix is very easy to interpret: It should be obvious why we call this a diagonal matrix. It has non-zero entries along the main diagonal and zeros everywhere else. It should also not take much to imagine what the linear transformation…
Robert Vagene's Blog 