Simplify do_experiment

[romainbrette]

Currently it makes scripts look a bit messy. First, one alternative mechanism (instead of adding a line on top of the script) would be with an import statement, with a python file that contains "do_experiment = True". But it's quite similar.

[romainbrette]

Actually a search and replace on do_experiment = True would be fine. Additionally, we could insert automatically at the beginning:

from clamper.setup.units import *
path = '.'

[mstimberg]

I have to say, I don't understand the reasoning behind this do_experiment variable in the first place...

The idea is to either 1) run the experiment + save the results + plot or 2) load the results + plot, right? Why not simply separate this into two scripts, i.e. one that runs the experiment + saves the results and another one that loads the results and plots them? Or, if for some reason it is important to keep everything in a single script, use something like do_experiment = os.path.exists('results.txt')?

But I feel like I might be missing the point here...

[romainbrette]

Yes I guess your last solution works. The idea is to copy the script that was run into the data folder, to know exactly what experiment was done. And also to be able to run the analyses directly in the data folder. But yes, I suppose we could have two files for each experiment, and the experiment script could run the plotting script at the end? In that case the plotting script needs to adapt to its current path (I guess with the same mechanism as you suggest).

[mstimberg]

Does everything has to be a script? The plotting code could be a function, no? I see that you want to have a convenience script that you can directly call from the data folder (instead of having one plotting script where you manually change a path), but this could be a simple wrapper around the plotting function.

For me, one major reason (apart from less code duplication, etc.) to have a plotting function instead of a script in each data directory is that if at some point you find a bug in your plotting code, or want to plot differently, you can simply change the plotting function and can do the plotting for all previous experiments instead of having to copy & paste stuff into the old directories.

[romainbrette]

Actually there is a little difficulty: the data folder name is generated by the experiment script, and so unknown to the other script. Makes it slightly more complicated.

[romainbrette]

Another issue is you might want to have intermediate plots. But that's feasible. Let's see if makes things clearer.

[mstimberg]

Actually there is a little difficulty: the data folder name is generated by the experiment script, and so unknown to the other script.

But if the plotting code is a function, you can pass the name of the directory as an argument. And in the "wrapper script" which is created in the data folder itself, it just uses the directory where it is.

[romainbrette]

Ok I've uploaded a working example!

[mstimberg]

Looks much better to me!

The analysis code should probably be in clamper.analysis or something, though, instead of in the examples folder? Oh, and since the examples folder is not a subpackage of clamper it should be at the toplevel of the directory, I'd say.

[romainbrette]

Why in clamper.analysis? It's specific of each experiment.

[mstimberg]

Why in clamper.analysis? It's specific of each experiment.

Ok, I see, sorry. You might still want to reduce redundancy a bit by putting commonly used stuff in a function somewhere, but if the basic analysis is different for each experiment, then it makes indeed sense to keep it in the examples folder.

[romainbrette]

Indeed, it would be nice to have some common stuff in there.