desihub / desisim

DESI simulations
BSD 3-Clause "New" or "Revised" License
16 stars 22 forks source link

Formalize basis_templates tagging procedure #554

Open weaverba137 opened 2 years ago

weaverba137 commented 2 years ago

Prior to basis_templates/v3.0, tags contained a checksum file. However more recent tags do not contain this file. Uploading and downloading very large binary files with svn is always a bit tricky, so the checksum file helps protect against corruption.

Here are some non-binding suggestions:

  1. Include the checksum file, but use sha256sum and call it, e.g., basis_templates_v3.3.sha256sum. That is, include the tag in the name.
  2. It would not be a bad idea to have a checksum file for trunk.
  3. Create some automation for preparing basis_templates/trunk for tagging, including creating the checksum file, modifying basis_templates_trunk.sha256sum as needed.
  4. Include the creation of the tag itself in the automation.
  5. Include a post-tag "bump" that restores basis_templates_trunk.sha256sum.

Here's some code that illustrates this:

cd basis_templates/trunk
rm basis_templates_trunk.sha256sum
sha256sum *.fits > basis_templates_trunk.sha256sum
svn commit -m "Update checksum file"
svn rename basis_templates_trunk.sha256sum basis_templates_v3.3.sha256sum
svn commit -m "Prepare checksum file for next tag"
cd ..
svn copy trunk tags/v3.3
svn commit -m "Tagging basis_templates/v3.3"
cd trunk
svn rename basis_templates_v3.3.sha256sum basis_templates_trunk.sha256sum
svn commit -m "restore trunk checksum file after tag"
sbailey commented 2 years ago

This sounds good. I suggest wrapping that in a script, perhaps kept in basis_templates svn rather than desisim, for tagging + checksumming basis_templates correctly. We could use that to make a new tag just to cleanup the checksum situation for the most recent tag.

weaverba137 commented 1 year ago

@moustakas, @sbailey, are there ever going to be new tags of basis_templates?

moustakas commented 1 year ago

I think it's not inconceivable that we revisit the desisim infrastructure and develop new basis templates going into desi-2 and beyond, so I wouldn't count it out.