Need of job script generator is a symptom

[martin-ueding]

I find the need of the job script generator a symptom that it is too hard to setup this program up for multiple configurations. I think it would make sense to have everything but the configuration number in a main configuration file and then do the various configurations in subdirectories. The code would either figure out the configuration number to use from the directory name or one would drop another configuration snippet there.

Though I like the

Convention over Configuration

motto from CakePHP, so perhaps this can be done here as well.

One downside, and this was addressed by Bartek already, is that the auditing trail is now a bit limited. One does not know exactly the parameters that were used for a particular configuration. Perhaps one has changed the parameters slightly and then just run part of the configurations? If one has one central file, one cannot have such a mixed parameter situation.

However, I don't want to have such a situation. And also, one could always just re-run the job script generator and overwrite all the parameter files without updating the contractions. So I don't really see the benefit of copying the parameters around.

So I would propose to have some structure like this:

contractions.ini
conf_1000/...
conf_1001/...
...

[martin-ueding]

I have added a little bit of code which prints out the configuration file to standard output before parsing it with boost program options. This way one can track what has happened. Also it is a needed step to get rid of the job script generator.

[martin-ueding]

The job script generator has been cut down significantly. It does not know about the contract specific command line options, just about the job scheduler specific ones. Therefore it does not generate configuration files for contract any more. Instead it will just create the job scripts such that they use one shared configuration file and have the configuration number in the command line.

The usage is now as follows:

• Create your contract.ini file with a text editor. Everything goes in there, all the quarks, all the operators, all the correlators. The configuration number should not be present there, but it will be overwritten anyway.
• Create some generate-jobs.sh script where you record your call to the generate-contraction-jobs generator. This way you can easily re-run and also have a audit trail. There just specify the machine specific options.
• Run the generated start-jobs.sh script and it will start the jobs as usual.

In order to have some audit, the shared configuration file is printed to standard output on startup of contract. This way it is possible to know from the log files what has been read in.

In the future the program should also nicely print out what it has read in, but this requires a bit more code. I perhaps do that after I have ripped this singleton class out of the code.