search

SETI / rms-pdsfile

pdsfile Python module
Apache License 2.0
0 stars 0 forks source link

Figure out how to handle PDS4 versions #31

Open rfrenchseti opened 5 months ago

rfrenchseti commented 5 months ago

Pds4File currently does not have a mechanism for handling versions of PDS4 bundles. We need to figure out how versioning is going to work (including whether files are copied or just symlinked), and then figure out how the rules in Pds4File should be constructed.

matthewtiscareno commented 5 months ago

@markshowalter's memo (attached, and also in Dropbox under RMS Policy Documents) is the canonical way to go. Under that system, versioned directories contain only the files that are superseded, not the files that remain unchanged in the current version.

However, since we don't have software tools in place to manage such a complex system, we are currently simply doing what we did for PDS3. This is to take the entire previous version and put it into a directory containing the version number.

rfrenchseti commented 5 months ago

Looking at the PdsFile code, it may just do the right thing with versions if we follow the same naming standard we used for PDS3. For uranus_occs_earthbased, we should not include the duplicated bundle, because it will show up all the time (versioning happens during download, not during search). Also we'll have to add a rule for the user's guide with the alternate naming.

matthewtiscareno commented 5 months ago

Sounds good. So do I have any further action items regarding this issue?

rfrenchseti commented 5 months ago

I guess one open question is whether you are versioning the bundles or the bundleset. Does everything go under uranus_occs_earthbased and there are now twice as many bundles? Or are there two bundlesets uranus_occs_earthbased and uranus_occs_earthbased_v1?

matthewtiscareno commented 5 months ago

Looking at Viewmaster, it looks like superseded volumesets reside in directories containing the version numbers, but their volume subdirectories do not. Perhaps do the same?

rfrenchseti commented 5 months ago

I think that would be cleaner as long as it meets whatever PDS4 standards we need to meet.

Do you have to change any LIDVIDs within the bundles?

matthewtiscareno commented 5 months ago

Part of creating a superseding bundle is fixing the LIDVIDs for the new version. We wouldn't update LIDVIDs in an old bundle. It is preserved as a product of its time.

rfrenchseti commented 5 months ago

Right. I just wanted to make sure that was being done for the new uranus_occs_earthbased bundle that is about to be created.

matthewtiscareno commented 5 months ago

I believe @mace-space is on the case with regard to that.

matthewtiscareno commented 5 months ago

(alliteration not intended) :)

mace-space commented 5 months ago

Trying to keep pace

rfrenchseti commented 4 months ago

It has now been decided that there is a superseded directory under the bundleset directory, like bundles/uranus_occs_earthbased/superseded. This directory should be ignored when enumerating bundles, but should be searched when looking for OPUS download products.