Closed HadrienG2 closed 6 years ago
Note that these ideas are based on my experience with verrou v1.1.0, but looking at the upcoming release notes, it looks like some of them are already in the process of being addressed.
Thanks a lot for your insight. For future reference, here are a few thoughts/comments about the issues you mention.
should be fixed by 8155d3d.
probably won't be addressed. One of the reasons why is related to point 6.: running all scripts in the same initial directory would make it difficult for a lot of programs to meet the "independent runs" requirements.
good idea. Although not planned yet, it will probably be addressed in a future release. Do you have any specific idea / use case in mind? There are a few detectable reasons why the cache should be invalidated (such as changes in the run_script or cmp_script). But there are also a lot of undetectable reasons: bad luck in random perturbations that wrongly makes a configuration valid, changes in any program or data file used in the run_script of cmp_script, etc. I don't see any way to handle such cases without asking the user to manually empty the cache.
verrou_dd
should probably not be responsible for running verrou, since (i) the user should be able to specify any command-line option they want, and (ii) not all commands in the run_script should be executed within valgrind/verrou. If I understand your proposition correctly, exporting a $VERROU
environment variable containing the correct command to run would be doable, but this would always contain something like valgrind --tool=verrou
, so that I'm not sure it would be very useful.
good idea. It is already in the TODO list for a future release (probably the release after v2.0.0).
should be at least improved by 8155d3d (but there might still be some ambiguous error reporting).
Regarding point 5 (verrou_dd
parallelization), parallelization at the scale of the global DD algorithm is not yet available (but is on the TODO list as mentioned above). However, commit 536569ff introduced a feature to enable parallelism at the scale of the test of a given configuration.
When the VERROU_DD_NUM_THREADS
environment variable is set, up to VERROU_DD_NUM_THREADS
tests are run in parallel to determine if a given configuration is stable or not. This may speed up the test for stable configurations (since VERROU_DD_NRUNS
runs must be performed in this case to validate the configuration). For unstable configurations, it does not save as much time: if a given configuration fails to validate the cmp_script at its first run, there would have been no need to launch many runs in parallel to get the same result.
Of course, if you have enough computational power, the optimal parallel setting is obtained when VERROU_DD_NUM_THREADS
=VERROU_DD_NRUNS
.
These remaining two issues start to stray a bit apart from the original ergonomics focus, so maybe it would be better to split them into two new tickets. Shall I do that?
Yes, please create new issues for points 3. and 5.
I believe 6525492 fixes the last ambiguous error message in verrou_dd
, so that we might be able to close this issue after that.
Besides #6 , another issue which I have with verrou_dd is that it is a bit difficult to use correctly. Here are some suggestions of quality-of-life improvements:
verrou_dd
should accept relative paths to the run_script and cmp_scriptverrou_dd
should run the scripts from its initial working directory in order to remove the need for cumbersome absolute paths inside of said scripts => Probably not a good idea, as applications may leave state around in the working directory and we aim for independent runs.verrou_dd
could be better at detecting when the dd.sym cache must be invalidated. => Moved to #9verrou_dd
's interface could be redesigned so that it is the one responsible for running verrou (or puts the proper valgrind command in an environment variable). This way, one would not need to repeat the valgrind command line in every run_script => Probably does not save enough characters to be useful, considering verrou_dd's requirementsverrou_dd
can take a lot of executions to converge, a way to state "my scripts are thread-safe, please fill up my idle CPU cores by testing multiple configurations in parallel" would be very nice. => In progress, parallel configuration testing is not there yet but parallel runs are in, see #8