In principle we can have different code versions for each process step, e.g., code version "extraction_v1.0.1" and "coaddition_v2.1.3" can coexist and they would just make different provenances.
The advantage is that if you update the code for a downstream provenance (e.g., for "measuring") you don't have to make new provenances (and new data) for all the upstream products like images, subtractions, etc.
The provenance model handles this ok, but there are a lot of places where we make the provenances, where we just grab the code version from the upstream provenance, or grab the "latest" code version. All these things will get more complicated, and probably require that we make some function that grabs the "latest" code version for a specific process step (is that going to be a new column on CodeVersion maybe?).
Also, if you don't want the "latest" version, you'd have to have a way to pass that requirement into the function that grabs the code version and be able to specify some other version. This can come up, for example, if you are deliberately trying to recreate a provenance tree of older data. It would not be enough to know the parameters, you'd have to specify the code version explicitly, too. I don't know how often this would come up...
guynir42
commented
4 months ago
In principle we can have different code versions for each process step, e.g., code version "extraction_v1.0.1" and "coaddition_v2.1.3" can coexist and they would just make different provenances.
The advantage is that if you update the code for a downstream provenance (e.g., for "measuring") you don't have to make new provenances (and new data) for all the upstream products like images, subtractions, etc.
The provenance model handles this ok, but there are a lot of places where we make the provenances, where we just grab the code version from the upstream provenance, or grab the "latest" code version. All these things will get more complicated, and probably require that we make some function that grabs the "latest" code version for a specific process step (is that going to be a new column on CodeVersion maybe?).
Also, if you don't want the "latest" version, you'd have to have a way to pass that requirement into the function that grabs the code version and be able to specify some other version. This can come up, for example, if you are deliberately trying to recreate a provenance tree of older data. It would not be enough to know the parameters, you'd have to specify the code version explicitly, too. I don't know how often this would come up...