VirtualPlantLab / PlantSimEngine.jl

A simulation engine for models related to plants
https://virtualplantlab.github.io/PlantSimEngine.jl/stable/
MIT License
15 stars 1 forks source link

Add parallel computations for time-steps in MTG (not just over nodes) #25

Open VEZY opened 1 year ago

VEZY commented 1 year ago

We compute the time-steps sequentially right now for each MTG node. It could be interesting to be able to add a parallel computation over it in addition to the parallel computation over nodes.

To do so, we have to change the way we compute the time steps. For now, we make a Status with one time-step, and we initialise it with the time-step, then run the simulation, then take the value to the attributes, and we do that for each time-step. This involves a lot of unnecessary copies, and is not parallelizable.

We have to find a better, more efficient and parallelizable way of doing that.

One proposition is to make a structure like the Status that would be a pointer to the values in the attributes. Users should initialize the attributes properly though, the length of each argument being of length of time-steps.

Another way would be to initialise the status properly with all time-steps, and keep the values there. And to avoid copies of the same values, the attributes in the node could be references to the values in the status. The benefit here is to use the status as it was intended, so we could use all methods from PlantSimEngine without any modifications. The inconvenient is that we have to find a good way to reference the values inside the status without changing the code elsewhere, and make it appear as it was just normal values.

One way or another, we need to define a structure to point to other values.

VEZY commented 1 year ago

OK so now the Status in the ModelList uses references to the values in the attributes. We also separate more properly the initialisation step from the simulation step, which clarifies the intent.

Now I'm thinking about implementing a method to put the attributes as TimeStepTables{Status} directly, and the ModelList in its metadatata. This way we wouldn't even need references, we would just pass the attributes to the ModelList, and that's it.

It would also be nice if this TimeStepTable construction could be node when parsing the MTG. To do that, we need to pass the models and the meteo to the read function.

VEZY commented 4 months ago

Many changes have happened since then; we now use a single Status for each node in the MTG, so parallelizing over time steps is no longer doable as is. What we should do now is to analyze the dependency graph, and if all models are time-step independent (and that it can happen in memory, i.e. not a very big scene), build a TimeStepTable of Status instead, and run the models in parallel over these. This would also allow merging all code between ModelList and multiscale models.