Open smilesun opened 6 years ago
It is allowed to "update" your problems/algorithms by simply re-adding them with the same name/id. This operation does not change any job ids, but only replaces the underlying code that is executed as soon as the job starts.
You're right that this is questionable w.r.t. reproducibility. However, I don't see how having a single job point to multiple result files could help here.
currently, in the results folder of the experiment registry, the results are named after i.rds where i is n integer. Suppose of the current scenario: the user needs to change some algorithms but still want to run them and put them in the same job id, in this case, will the results rds file be overwritten? If so, is it better to store using the job hash id of the algorithm and problem and maybe git sha key?