ID_simul management

[Emblanc]

WALTer.lpy generates an ID for each simulation and writes it in a file in the simulation directory The ID should be generated by the launcher

[pradal]

Add a column with the ID for latter analysis

[pradal]

Analysis:

• The project do not generate ID for the simulation
• An ID is used to keep the relation between the input parameters and the output directory where all the files are written.

To implement the new strategy:

• generate a file that contains the dict with:
  • key: a new ID per set of parameters (simulation)
  • values: the parameters (dict)
• generate the same file named from_json_to_humanbeing.txt (but stored in columns)

[Emblanc]

generate the same file named from_json_to_humanbeing.txt (but stored in columns) :+1:

This is exactly what we need to be able to analyse the outputs of our simulations : we used to have such a file (but it was generated by a bash script) and all our analysis scripts are based on it. Ideally, the new file should be a csv file with tabulation separator. The file should be the same one as the sim_scheme.csv used to launch the simulation(s) but with an extra column named ID. We could call it combi_param.csv.

[pradal]

Nice. This is what we have done. Do you need ID_simul.txt file? The problem is that if we run several simulation in parallel, this file is written by several processes.

Another solution may be to pass the ID as parameter, or to run each simulation in its own repository with inputs and outputs. Something like: hufdjkghkjhgd/ input/ output/

Then we do not care about unique ID but we can manage the simulation number after the simulation.

[christian34]

A small update from today's Bilal activity: the combi_param.csv has been generated and ID is being passed to walter.lpy via params (he will probably comit tomorrow)

Still, from within python, this require several manual steps (that are chained in main.py):

• create a project with directory name as input
• read parameters with parameter file as input
• create id: param table with parameters as input
• run simulation(s) with id:params as input
• eventually, in tests, inpect output with id as input

To automatise things a bit further, the first 3 steps could be done at project instanciation, and then run will run all (default) or some (identified by number) simulation. This could work if 1 project =1 directory = 1 csv parameter file = xx simulations (one per line of the csv) can we consider we are always in this the case (or does the csv file attached to one dir evolve with time ?)?

[pradal]

Have you seen my branch where I implement all of this? This is the issue #46 (only some part of what you present).

[christian34]

Yes this is the branch he worked on

[Emblanc]

Nice. This is what we have done.

Perfect !

Do you need ID_simul.txt file?

I do not need this file. It used to be written just to pass the information of the id from WALTer.lpy to our bash script.

Another solution may be to pass the ID as parameter, or to run each simulation in its own repository with inputs and outputs. Something like: hufdjkghkjhgd/ input/ output/

It is better for me if it stays the way it is now with : my_project_containing_several_simulations/ ..output/ ....nbovefbvef/ #outputs for simulatoin1 ....laqoscdnso/ #outputs for simulation2 ....

To automatise things a bit further, the first 3 steps could be done at project instanciation, and then run will run all (default) or some (identified by number) simulation. This could work if 1 project =1 directory = 1 csv parameter file = xx simulations (one per line of the csv) can we consider we are always in this the case (or does the csv file attached to one dir evolve with time ?)?

We can consider we are always in this case : each time I need to run a set of simulations, I open a new project.

[pradal]

Excellent. So we are quite close to the result. We mainly just have to investigate why we have some times errors.

[christian34]

Closed by #46