net.connectivity data structure

[jasmainak]

The current net.connectivity data structure is not super readable and it's not efficient either. We should rethink this a bit. I propose that we start with this:

net.connectivity['L2Pyr -> L2Pyr'] = [ {'src_gid': 43, 'loc': 'proximal', 'receptor': 'nmda', 'A_weight': 0.0005, 'lamtha': 3.0,'target_gid': 45}, ...]}

This already saves you repeating the cell type and is also a bit more organized. This is what I tried for object size:

from mne.utils import object_size

size = object_size(net.connectivity) / (1024 * 1024)
print(size)

which gave me 55 MB????

ps: I am not sure if we want to allow changing anything other than 'src_gid' and 'target_gid' to vary between two cell populations? Might be worth considering bringing out as many parameters as possible into the outer dictionary

[jasmainak]

cc @ntolley

[ntolley]

Yikes that definitely needs to be cut down. I'm not sure if I want to give up the ability to modify the connection details, but I see no reason for this to be included for the default behavior.

Here's an idea, a single dictionary could store the "default" values for the different classes of connections. But for more fine grain control, a flag like net = Network(params, unique_connections=True) creates the more redundant form of net.connectivity.

[jasmainak]

I'm not sure if I want to give up the ability to modify the connection details

You still will be able to modify the connection details. I guess the question is what level of granularity do you want? Individual cell-to-cell, population-to-population, or cell-to-population? Even if you do population-to-population but store the weights as a matrix, you save repeating the parameters such as threshold, loc etc. which I'd say YAGNI for modifying on a cell-to-cell basis.

a single dictionary could store the "default" values for the different classes of connections.

I wouldn't advocate for too many different code paths. It's the perfect recipe for bugs. We should reuse the API as much as possible, even internally in the code. Try to think of alternate data structures that reduces repetition

[ntolley]

After thinking about this for a few days, I think I've settled on a structure that preserves the ability to specify individual connections while dramatically cutting down the size of net.connectivity.

At the moment there are 16 unique connection types. For the default matrix, net.connectivity could be a 16 element list of dictionaries with the format:

{'src_type': 'L2_basket', 'target_type': 'L2_pyramidal', 'receptor': 'gabaa', 
 'weight': 0.0005, 'connections': [(src_gid1, target_gid1), (src_gid2, target_gid2), ...]}

@jasmainak @cjayb let me know what you guys think!

[jasmainak]

~Looks good. But 'weight' above should also have two elements?~

Edit, okay maybe not. This looks great to me!

[jasmainak]

Time to make a PR I'd say. I like this :) Also you might want to develop a repr from this. Something like:

print(net.connectivity)
<L2 basket -> L2 pyramidal (ampa, gabaa) 105 connections
L2 basket -> L5 basket (blah, blah) xyz connections
>

[cjayb]

This looks like a good step @ntolley The cool thing is that the tests you made in #276 should catch discrepancies you run into when making this into a PR, right? ;) Looks to me like 'connections' could become quite sparse for the local L2_basket connections.