iterative / dvc

🦉 Data Versioning and ML Experiments
https://dvc.org
Apache License 2.0
13.95k stars 1.19k forks source link

"appending" dvc.log.lock file type idea for exps #10612

Closed majidaldo closed 2 weeks ago

majidaldo commented 2 weeks ago

I've been using dvc for a while and I understand the architecture and philosophy. I also understand the decisions behind dvc experiments with the challenge there being trying to align git's linear development notion with experiment's more "parallel" notion.

I think having to go through dvc exp is too "heavy". Now, if you think of using special git refs as "backend" storage for experiments, alternatively you can achieve the same with dvc lock files that append (instead of overwrite) runs. This way you can keep original (simple) dvc behavior of just dealing with lock files while also capturing the same information for parameterized runs (exps).

So, I should be able to "checkout" a particular run from the dvc.log.lock file instead of having to deal with dvc exp (parameterized or not).