Structural plasticity

[thesamovar]

We should add structural plasticity as an option, but we need to consider how exactly to do it. One case we considered is relatively simple to implement and could probably be done with minor changes to existing code, then we would just need a few examples of it being used to point users towards. This idea is that you would do structural plasticity operations infrequently, say every minute of simulated time. You would do a pruning operation, e.g. remove all synapses whose weight had dropped below a threshold, and a creation of new synapses operation, i.e. adding new random synapses to replace the pruned ones. To make this useful, we need to think about a string syntax for modifying an existing set of synapses, e.g. S.delete('abs(w)<w_thresh'). It may be tricky to make structural plasticity play nicely with standalone.

[mstimberg]

I think the situation here has not changed much, in the sense that we are still unsure about the typical models that are used, hence about the appropriate syntax for creating/deleting synapses during a run. But I wonder whether we should not add a delete (or maybe disconnect?) method pretty soon? It would be a mirror of the connect function (i.e. you could give a string expression or a a list of sources/targets, probably no need for the p and n arguments) and is pretty straightforward to implement within our current framework (including standalone).

Even apart from the structural plasticity use case, it might sometimes simply be more practical to create too many synapses and then delete some, e.g. do something like

S.connect(True)
S.disconnect([3, 7, 8], [4, 8, 9])

[thesamovar]

I agree, and I like your name disconnect.

What I'm wondering, is whether or not we could put operations like connect/disconnect on a clock? That's what you'd need for proper structural plasticity. I see two ways to do it: (1) put it on a clock, (2) support loops somehow in standalone. I like both but both are potentially complicated, it would be fine to think about that later.

[svadams]

Hi All,

(apologies for the long post, bit of a brain dump!)

Dan directed me over here from the main mailing list as I worked on structural plasticity simulations using Brian 1 quite a while ago and would like to return to this area of research, so I have an interest in seeing this implemented in Brian 2. I would be very happy to contribute to development and testing as well.

In my Brian 1 implementation pruning was pretty straightforward - just deleting all synapses falling below a threshold - however I only ever did this by checking connections between neurons that actually fired. Having a means to check all synapses on a clock running at a less frequent interval than the main simulation would be really useful. Probabilistic pruning was also something I never got around to doing but wanted to. Here any synapse could be pruned with probability p rather than by its weight.

My synaptogenesis was a bit more involved (and inefficient!) as I was looking at activity dependent synapse creation. Again I considered only firing neurons and created new connections between them according to distance and pre/post firing times. However probabilistic creation would also be a very useful thing.

I like the idea of adding a S.disconnect() to the API to enable synapse deletion. Having full flexibility to call connect / disconnect to change the connection matrix during a run would be the ultimate feature for me but in the short term having more specific limited functionality would be a good step. My order of preference would be:

Synapse pruning controlled by a clock and either a weight threshold or probability.

A synapse pruning monitor to record a count of synapses deleted or the pre-post neuron ids of deleted synapses

Synapse creation controlled by a clock and a probability for creation (also there would have to be a means to specify the initial weights of new synapses)

A synapse creation monitor to record a count of synapses created or the pre-post neuron ids of created synapses

Regards Sam

[thesamovar]

Thanks Sam - it's really useful to have these user perspectives for when we decide on a design!

I'm wondering if we did the following would it cover a large amount of what was needed:

• Add the ability to call connect on a clock. You could already do this with a network_operation but then it wouldn't work well with standalone mode.
• Add syntax to connect to only add a synapse if there isn't already one, e.g. something like a boolean variable connected that tells you if there is already a connection or not. What are the computational implications of this I wonder?
• Add a disconnect that works similarly to connect in that you can be explicit or give string conditions, etc.

I think we could do a lot with this. Thresholding would be simple with disconnect on a clock. Activity dependent creation could work if you added a variable to the source/target NeuronGroup to estimate firing rates. Probabilistic creation would be as it is at the moment.

@svadams would that cover all your use cases?

@mstimberg what do you think? This seems quite general and flexible, and relatively easy to implement. It also wouldn't stop us from doing something more complicated in the future if we decide to.

[svadams]

Yes, this sounds very good. Definitely only want to create a synapse if there isn't one already - I used to do it by checking if the value of the connection was an integer value. Not sure if that was a Brian thing or a numpy sparse matrix property. I think it would be better to have an explicit 'connected' property to check though. Maybe you could have checking for existing connections as an optional parameter in the call to connect then by default no checking needs to be done and the user only enables it when they explicitly mean to add new connections during the simulation?

[mstimberg]

Good discussion! I think the easiest thing to do in the short term is to add the disconnect function -- I don't think there's much to discuss syntax-wise and also the implementation is straightforward (very similar to the connect templates). The only detail worth discussing is how to handle the situation that the user requests to disconnect non-existing synapses. I'd say that for expressions like i != j this is not actually an issue, we'd only check the existing connections anyway. On the other hand, for something like S.disconnect(i=[1,2,3], j=[3,2,1]) it might be good to have an error message if one of those connections does not actually exist (similar for the generator syntax)? My take on this question: let's re-use the skip_if_invalid keyword argument we have in connect -- i.e. raise an error if it is set to False, ignore such errors if it is True.

About putting connect/disconnect on a clock -- this is actually quite straightforward with our framework. I think for it being maximally useful we'd probably want to implement #738 first, because often you do not only want to create the synapses but also set the initial values of their weights/delays/etc. And I agree that we need a syntax to restrict it to existing connections. @svadams' proposal sounds like the easiest to implement, e.g. we could have a skip_if_exists keyword?

[thesamovar]

Agreed! Sounds like we have a pretty solid proposal for this.

[svadams]

@mstimberg yes I like the idea of using skip_if_invalid for disconnect as well. I also just looked at #738 and think that is going to be essential - I would always want to set weights and delays on a new connection, and it would be very useful to be able to vary that during a simulation (for example, change the probability of connection)

[galharth]

Hi Guys, any update regrading the disconnect feature? I'm working on a learning algorithm who like the brain needs to prune/create new synapses on the run (while learning). Any hack/alternative to do it with the current version? Thank you very much 🙏

[svadams]

Just to add - as one of the people asking for this I'd be very happy to contribute some development effort to begin to get some of this functionality added. If the disconnect implementation is based on existing connect one as we discussed above then I should only need some basic guidance. Although ideally #738 would be already done I think I could get started on disconnect.

[thesamovar]

We would still love to implement this feature but the Brian devs are all a bit tied up with writing up a couple of Brian papers at the moment, so adding new features might need to wait a bit. Would be great to get help, but I suspect that this will be one of those cases where you would need a lot of knowledge about how Brian internals work to contribute. If you do fancy forking the repo and having a go though, we'd be very happy! :)

[svadams]

Appreciate  this is probably a niche feature so not at the top of your list Dan! I would love to at least have a look as I'd like to know more about how Brian works in any case.

[dmnburrows]

hey guys, wondering if you had any update on adding this disconnect feature?

Thanks!