Is your feature request related to a problem? Please describe.
intermediate data structures are pickled to allow them to be loaded if the run is interpretted. however a user can rerun with different parameters. currently the parameters from previous runs are not compared and major problems could arise if checkpointed files are used with different parameters. some values can change (eg. num threads) but other values need to remain consistent.
Describe the solution you'd like
the parameters should be pickled at the start of a run and compared to the parameters of the new run. a checkpoint should occur only if they match. if they don't ask the user should be warned that the existing pickles will be overwritten unless they use compatible inputs.
it also would be prudent to log the user's input command. should be pretty simple to implement (" ".join(sys.argv)).
Is your feature request related to a problem? Please describe. intermediate data structures are pickled to allow them to be loaded if the run is interpretted. however a user can rerun with different parameters. currently the parameters from previous runs are not compared and major problems could arise if checkpointed files are used with different parameters. some values can change (eg. num threads) but other values need to remain consistent.
Describe the solution you'd like the parameters should be pickled at the start of a run and compared to the parameters of the new run. a checkpoint should occur only if they match. if they don't ask the user should be warned that the existing pickles will be overwritten unless they use compatible inputs.
it also would be prudent to log the user's input command. should be pretty simple to implement (
" ".join(sys.argv)).