Plasticity as a way to compute z-score

[tcstewar]

There are some models of unsupervised synaptic plasticity that are thought of as the neurons trying to adjust their parameters such that they fire a certain proportion of the time. This is sometimes thought of as the neurons acting as a function that takes some input distribution of spikes and outputs a more uniform distribution (or gaussian, or some other distribution).

If we put that into an ensemble (with each neuron still having its own encoder, and having its own desired output distribution), then we might get an effect that is something like transforming x into the z-score for x (the number of standard deviations away from the mean).

[tcstewar]

(Johannes Leugering is a student at Osnabruck who just completed his Master's thesis on this sort of plasticity and is interested in seeing what this sort of model would do at the population/vector level. The title of the thesis is "Adaptation of Neuronal Activation Functions to Arbitrary Distributions of In- and Output")

[tcstewar]

And it looks like Johannes has gone ahead and made a full Nengo NeuronType plugin that does this!

https://github.com/jleugeri/nengo-adaptiveLN-models

It works beautifully. Here's a quick model that uses it and lets you adjust the gain and bias on a sine wave input so you can see it adapt:

import nengo
from nengo_adaptiveLN_models import *
import numpy as np

model = nengo.Network()
num_neurons = 1000

def stimulus(t, x):
    bias = x[0]
    scale = x[1]
    return np.sin(t*2*np.pi)*scale + bias

position = np.random.randn(num_neurons)*0.5
scale    = 0.5+np.random.rand(num_neurons)

#neuron_model = AdaptiveLNuniform
neuron_model = AdaptiveLNlogNormal

# Set time-constant of the adaptation variables in seconds
tau_adapt= 10.0

with model:
    ctrl = nengo.Node([0.5,0.5])
    stim    = nengo.Node(stimulus, size_in=2)
    nengo.Connection(ctrl, stim, synapse=None)
    ens = nengo.Ensemble(n_neurons=num_neurons, dimensions=1,
                            neuron_type=neuron_model(tau_adapt=tau_adapt, position=position, scale=scale),
                         bias=np.zeros_like(position), gain=np.ones_like(scale))

    nengo.Connection(stim,ens, transform=1, synapse=1e-5)

Right now the only weirdness is that it's using an internal gain and bias value, rather than the one specified in the Ensemble.