Putting the Neural back into Networks
Part 3: I got 99 problems but a spike ain't one
13th January, 2020In the last post, we saw that spike generation can cause major issues with gradient descent, by completely zeroing the gradients. We also learned how to get around this issue by faking the neuron output by making a spiking surrogate.