astoeckel commented 6 years ago

Note: This issue is for brainstorming regarding neuron models with multiple input channels and the related issue of synaptic computation. Implementing this would require major restructuring of Nengo's internals ‒ this issue is purely hypothetical and explores how these features could be exposed to the user. By no means do I intend to implement this outside of my scientific experiments at the moment.

Note: Feel free to edit this issue if you have any additions. This should allow to keep suggestions in a central place. Still, it might be wise to add a short comment below.

Motivation

What are multi-channel neuron models?

Multi-channel neuron models are neurons with dynamics depending on more than one input variable. Currently, the NEF/Nengo restricts neuron models to depend on one input variable, the input current J. J is hard-wired into Nengo's logic (e.g. J is automatically multiplied by gain and bias).

Unfortunately, biologically more plausible neuron models have multiple input channels; none of which may be a current. For example, simple neuron models with conductance based synapses possess non-negative excitatory and inhibitory input channels GE and GI with independent synaptic filters. Complex multi-compartment neuron model possess a variety of conductance based synapses with varying dynamics (i.e. different filter time-constants). Other examples include inputs modulating synaptic adaptation, hormones/neurotransmitters, etc. ‒ everything that could change the properties of the neuron while the simulation is running.

Why is this nice to have?

There are multiple reasons why this is a potentially nice feature to have in Nengo:

Biological plausibility: Allows people to better constrain their models to biological evidence. E.g. explicitly modelling non-negative excitatory/inhibitory channels allows to better account for Dale's principle. Conductance-based synapses in general can give a very good sense of the biological plausibility by directly exposing biological units (reversal potentials, conductances).
Neuromorphic hardware and external neuron simulators: Some neuromorphic hardware (Spikey, BrainScaleS) implement conductance-based channels. Additionally, simulators like NEURON could be interfaced more naturally.
Greater computational power: Conductance based synapses allow for synaptic computation. Non-linear multivariate functions can be computed without ever representing the input variables in a common neuron population. This reduces the number of required neurons. See below.

What is synaptic computation?

Suppose you want to calculate a product xy in Nengo. According to classical NEF rules such a function can only be computed if both x and y are represented in a single neuron population as a vector (x, y). The connection from this central population to another population can then compute a multivariate function xy.

Put differently, when connecting from two populations A1, A2 to a population B, the classical NEF formalism dictates that B must represent a linear combination of the values represented by A1 and A2. I.e. if A1 represents x and A2 represents y, B must represent x + y (modulo scaling).

Synaptic computation exploits nonlinear dependencies between neuron input channels to perform computation. The interaction between the inputs from the pre-population results in effective somatic currents J that are equivalent to the neuron population representing xy.

Note: To work effectively, synaptic computation requires (simple) multi-compartmental neuron models (two compartments). It does not work well with the simpler IfCondExp neurons found in other simulators. So yes, effectively we still have two neuron layers (the dendritic and somatic compartments), but both compartments are logically/physically one neuron.

Why is the implementation of this hard in Nengo?

J is currently hardcoded in Nengo. Implementing these features would require to rip everything apart, including things such as the graph optimizer and all backends.
Concept of gain and bias does not exist for conductance-based synapses (gain and bias only determine the desired neuron tuning curve that is being solved for, and not any model parameters)
Without gain and bias dense weight-matrices are required everywhere (each post-neuron has its own decoding vector for each channel). However, often, the weight-matrices can be factorised into lower-dimensional encoder/decoder matrices without loosing that much precision.
Unclear how to expose to the user. API changes should not break current code, and not (significantly) increase the number of primitives.

API proposals

Synaptic computation and multi-channel neurons are two independent issues. However, synaptic computation depends on multi-channel neurons.

Multi-channel neuron models

Optimally, the user should not have to care about the number of channels in their neuron models. Just selecting a multi-channel neuron model should make things work magically. E.g. given that LIFCond is a conductance-based multi-channel neuron

ens_1 = nengo.Ensemble(10, 1, neuron_type=nengo.LIFCond())
ens_2 = nengo.Ensemble(10, 1, neuron_type=nengo.LIFCond())
nengo.Connection(ens_1, ens_2)

should work out of the box, including interaction with other neuron types. This must also be true for Node connections.

Things change when explicitly targeting the .neurons object. The following must not work:

ens_1 = nengo.Ensemble(10, 1, neuron_type=nengo.LIFCond())
ens_2 = nengo.Ensemble(10, 1, neuron_type=nengo.LIFCond())
nengo.Connection(ens_1.neurons, ens_2.neurons) # Error: which channel is targeted?

When targeting neurons, one must specify which channel should be targeted. In this case, the following might work:

ens_1 = nengo.Ensemble(10, 1, neuron_type=nengo.LIFCond())
ens_2 = nengo.Ensemble(10, 1, neuron_type=nengo.LIFCond())
nengo.Connection(ens_1.neurons, ens_2.neurons.excitatory)

Note that the output of a neuron does not change ‒ it is still just spikes ‒ so a channel must not be specified on the pre-population neurons object.

Probes should be able to probe the channels:

nengo.Probe(ens_2.neurons.excitatory)

Discussion: Should it be possible to target channels directly with decoded output? I.e. something like

ens_1 = nengo.Ensemble(10, 1)
ens_2 = nengo.Ensemble(10, 1, neuron_type=nengo.BioNeuron())
nengo.Connection(ens_1, ens_2.dopamine)

I (Andreas) would not allow this, since this would totally screw around with units ‒ decoded values are not physical quantities (yet channels are; think conductances, currents, neurotransmitter levels). Conversion should explicitly go through the neurons object to indicate that “something special is going on here”, just the way it is right now if you want to target the current of a neuron. Furthermore, the population as a whole may not have a single value for that channel, i.e. each neuron has different conductances at a given point in time. This may be different for more global channels (neurotransmitters).

Synaptic computation

Synaptic computation could for example be implemented by allowing a tuple of sources for the pre field of a connection:

ens_1x = nengo.Ensemble(10, 1)
ens_1y = nengo.Ensemble(10, 1)
ens_2 = nengo.Ensemble(10, 1, neuron_type=nengo.LIFCond())
nengo.Connection((ens_1x, ens_1y), ens_2, function=lambda x, y: x * y)

Note that this should also work for target ensembles that do not support synaptic computation (e.g. standard LIF neurons). The math used for solving is the same, it will just not work properly.

Also note that in the case of conductance-based synapses the default behaviour of

nengo.Connection((ens_1x, ens_1y), ens_2, function=lambda x, y: x + y)

is not a special case. Since the synapses are nonlinear, the same math as in the x * y case must be used to figure out how the synaptic non-linearity must be countered. However, Nengo should hide this from the end-user and just perform addition whenever multiple connections target the same post-population.

Dale's principle

With the introduction of excitatory and inhibitory channels it would be nice to also have excitatory and inhibitory neurons, i.e. neurons that only project inhibitorily/excitatatorily. Mathematically, this is trivial to implement with the math for synaptic computation.

For example, it would be nice to be able to mark the fraction of neurons in a population that are excitatory/inhibitory like in the following example

ens = nengo.Ensemble(10, 1, roles={"excitatory": 0.7, "inhibitory": 0.3})

Similarly, one might want to have neurons that only interact with the endocrine system, i.e. that only project onto special channels; yet this seems to be a modelling-choice that must be expressed by the user and is not something that is immediately computationally relevant.

Mentioning @tcstewar and @psipeter, who can probably add some thoughts to the discussion.

jgosmann commented 6 years ago

Implementing this would require major restructuring of Nengo's internals ‒ this issue is purely hypothetical and explores how these features could be exposed to the user.

Haven't read the rest of the issue yet (but looking forward to it!), but this seems like the Nengo Enhancement Proposal (NEP) repository might be a better place for this?

jgosmann commented 6 years ago

I proposed to introduce named error signals for learning rules before (#1307, it was decided to revisit this issue someday after the builder refactoring). It seems that there is a similar need for named inputs for the conductance based neurons. I think having a system for such named inputs that could be redeployed where needed (learning rules, conductance based neurons) would be great. The code from #1307 (especially 3deaab9082d7686dd5ef99231cf9dcbd756622fa) might be a good starting point.

Nengo should hide this from the end-user and just perform addition whenever multiple connections target the same post-population.

Could there be cases where one just wants to investigate the resulting behaviour when no specific equation to compute is given? (Not sure if that even makes sense in principle.)

For example, it would be nice to be able to mark the fraction of neurons in a population that are excitatory/inhibitory

Personally, the proposed syntax does not seem to fit well with the existing syntax.

tcstewar commented 6 years ago

Haven't read the rest of the issue yet (but looking forward to it!), but this seems like the Nengo Enhancement Proposal (NEP) repository might be a better place for this?

My feeling is that this Issue is just meant to be a very speculative "hey, we should think about this" and I almost feel that it's so tentative that it's not even a proposal yet. But I don't remember what our policy was for how developed something should be before it's a NEP. I'm happy to have the discussion here or in the NEP repo, but my inclination is to have it here just because it's such early stages of brainstorming possibilities....

I think one thing I'd like out of the discussion is even just identifying what things are really hard to do in Nengo right now that are related to this, and hoping that leads to some alternate syntax options that we would want to take into account. Because the current Nengo design makes some pretty strong assumptions that @astoeckel 's conductance stuff is challenging, and that might require rethinking a number of aspects of Nengo's design.....

astoeckel commented 6 years ago

Could there be cases where one just wants to investigate the resulting behaviour when no specific equation to compute is given? (Not sure if that even makes sense in principle.)

Hm, that is a good point. I think that this would be confusing to the end-user as a default behaviour (since the story we usually tell is that projecting onto the same post-population implements addition), but I agree that there might be circumstances under which you want to tell Nengo to “just add these channels” and do not care about the computation that is being performed.

In the proposed API that could be achieved by manually calculating the connection weights and targeting the .neurons.excitatory and .neurons.inhibitory objects.

My feeling is that this Issue is just meant to be a very speculative "hey, we should think about this" and I almost feel that it's so tentative that it's not even a proposal yet.

Fully agree. I just tried to fill out the NEP template, and everything I wrote felt to vague to be a concrete proposal.

arvoelke commented 6 years ago

I like the proposed syntax:

nengo.Connection((ens_1x, ens_1y), ens_2, function=lambda x, y: x * y)

Nifty!

We may also want some notation to factorize the weight matrix and throw away the lesser modes for computational efficiency, as you suggested. Maybe even an additional argument to the Connection that provides a generic matrix factorization callable for approaches including SVD. Perhaps a triangular decomposition is more efficient on an some hypothetical hardware architecture? ¯\_(ツ)_/¯ lol

Practically speaking, it might be easiest to fork Nengo, see everything that needs to be changed, and experiment with that for a while. Since this eliminates the fear of needing to cooperate with existing tools/backends in the Nengo ecosystem, it shouldn't be toooo much work. Just my two cents.

astoeckel commented 5 years ago

Just to let people know, I've started to implemented parts of this in https://github.com/astoeckel/nengo_bio

nengo / nengo

API for multi-channel neuron models/synaptic computation #1410