move domain decomposition into model construction

[bcumming]

Currently domain decomposition is performed before model construction, and a simple partition of the cell ids across domains is passed into the model constructor.

To handle different cell types, internal to external gid mappings, and load balancing of cells with different computational weights, the model constructor will have to handle domain decomposition.

This task performs the first step: moving the domain decomposition logic that is currently implemented (a naive partition of gid across the domains) into the model constructor.

[halfflat]

I think there is room to split the domain decomposition separate from the model constructor, unless we're anticipating dynamic load rebalancing. It would keep separate the separate concerns, which I believe would help keep the code more modular.

We'll need some gid permutation representation regardless, for translating to and from external gid names.

The model constructor could take the recipe, the permutation and a (permuted) gid range, and do pretty much what it does already. If the load balancer is going to pick cell group sizes, then instead of a gid range, the model constructor could take a list of grouped gids, one per cell group.

The load balancer then can act independently of the model code; it just needs to be able to interrogate the recipe object and (of course this is the non-trivial bit) decide on the appropriate grouping and partitioning, given the operating environment.

[bcumming]

This is a good point, and I think you have got the key design decision there.

I am implementing a separate "load balancer" (not a very clever one). The logic for load balancing, group partitioning and in future gid permutation logic is not dumped into the model constructor. It is modular, so you could easily create it inside or outside of the model. It performs the role you described above.

I was wondering about the right place to create and store the domain_decomposition object (as I have tentatively named our load balancer).

It can be done before constructing the model, and passed as an argument to the model constructor, or it can be constructed in the model constructor and be part of the model state.

I lean towards keeping internal to the model, because information inside the domain decomposition isn't useful outside (as far as I can see) the model. When a user interacts with the model they will use the "user space" gid, and the model should perform the translation between user space and its own internal ordering.

Currently in the miniapp the cell_range and group_partition information is calculated, then passed into the model, and is not used elsewhere (except for the decidedly dodgy injection of spikes to initialize the model).

[w-klijn]

Regarding the name: after the function it has, or what it does? because domain_decomposition does not say me anything.

Regarding dynamic load balancing: In the far future (eg >5 years) we might arrive at developmental models where we might need to balance the load dynamically. Picking a design that fundamentally makes it impossible might not be smart.

Regarding optimization: I gave Ben a short introduction to Model creation in NEST. One of the discussed topics was the 'Poisson' input to 'normalize' the neuron membrane potential. Each neuron gets a unique spike train injected at a very high rate (eg. 8000hz) to model neurons NOT part of the simulated neurons. You would typically have 1 such neuron per 'real' neuron in your model. You do not want these spikes on the global spike exchange. You do want to have them on the local exchange to allow reuse of the GPU communication etc. Does the current design allow for these concideretions? eg: The load balancer should be able to place the Poisson neuron at the node with only local targets. (ohoh I used the word TARGETS)

[halfflat]

In this particular case, where spikes are being used to produce a noisy base line for membrane potential, I really think we'd be better off making this a particular point mechanism. Mechanism instances are intrinsically tied to their cell, so we avoid the problem of locality, and it avoids the problem of handling these spikes entirely.

It's also IMO a better conceptual fit. We need to support random functions in mechanisms regardless.

[bcumming]

I agree with @halfflat that for the specific use case @w-klijn needs for his model, a point mechanism seems to be the best fit.

"Domain decomposition" is a widely used HPC term for breaking your model (domain) into distributed parts. Typically it is used for things like fluid simulations where you break the 3D area that you are modelling into smaller chunks, with 1 chunk per MPI rank.

I am open to a new name.

[w-klijn]

In NEST it is implemented more akin to a point process. But from a 'model' description it is a true neuron.

But we need to get stuff running. I think we should take the neuron aprough for now and if performance suffers go for a faster implementation: The problem is that with the fast implementation you will have to have specific implementations and you make it allot more complicated.