Putting the Neural back into Networks

Part 1: Why spikes?

11th January, 2020

Not long ago, one of the gods of modern machine learning made a slightly controversial statement. In the final slide of his ISSCC 2019 keynote [1], Yann LeCun [2, 3, 4] (that’s “Mr CNN” to you) said he was skeptical about the usefulness of spiking neural networks, as almost a throwaway remark.

Schroedinger's importer

27th August, 2018

A Python package I'm working on combines several submodules with mixed licensing — some will be open source and redistributed, others will be proprietary and in-house only. I wanted to ease importing of the package by automatically detecting which submodules are present, and dynamically importing only those.

Visualised behaviour of dynamic neural networks

27th July, 2017

Behaviour of a random two-neuron dynamic neural network.

The dynamics of simple nonlinear systems can be dramatically beautiful. I visualised a large number of randomly-generated systems as an exploration.

Stochastic descent optimisation in Matlab

Using the Adam optimiser

21st February, 2017

Stochastic gradient descent is a powerful tool for optimisation, which relies on estimation of gradients over small, randomly-selected batches of data. This approach is efficient (since gradients only need to be evaluated over few data points at a time) and uses the noise inherent in the stochastic gradient estimates to help get around local minima. This is a Matlab implementation of a recent powerful SGD algorithm.

Putting vision into context

And adding context to vision

21st December, 2015

This optical illusion shows how strongly the context of a visual scene influences what we see, and how that context can sometimes mislead us. The three people in the picture are exactly the same size. However, because our brain judges the size of an object based on its perceived distance, the person at the back seems to be further away and therefore appears larger.

Our brain does not faithfully interpret visual information, but instead uses a complex mixture of prior experience and context to shape our perception of the world. We described a pathway in the brain for this contextual information, from the mouse thalamus to visual cortex.