Support hash-only build paths

[nkaretnikov]

Fixes #678.

Description

This pull request:

• adds hash-only build paths (v3)
• updates a test
• updates documentation.

Pull request checklist

• [x] Did you test this change locally?
• [x] Did you update the documentation (if required)?
• [x] Did you add/update relevant tests for this change (if required)?

Additional information

How to test

Automated testing:

See tests changed in this PR.

Manual testing:

• [x] with ~/.conda-store containing builds created on main, switch to this branch and create a new build: old and new builds should be accessible in the UI (lockfile, archive, and log links should work; marking one of the old builds as default should work)
• [x] same, but also explicitly set build_key_version = 3 in the config
• [x] same, but also explicitly set build_key_version = 2 in the config (make sure the previous default works with this PR)
• [x] make sure two exact same specifications can be created in different namespaces and one of them is preserved when the other one is deleted.

[netlify[bot]]

✅ Deploy Preview for conda-store ready!

Name	Link
🔨 Latest commit	806da5f0cba46bdef8e1dcb5edf10e34c2922fa1
🔍 Latest deploy log	https://app.netlify.com/sites/conda-store/deploys/66319c616dfacf0008a939a3
😎 Deploy Preview	https://deploy-preview-777--conda-store.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[nkaretnikov]

@jaimergp PTAL. Keeping as a draft for now because this needs to be addressed first: https://github.com/conda-incubator/conda-store/pull/777#discussion_r1519165915. Will do final testing once that is resolved as well.

[nkaretnikov]

@jaimergp This is tested and ready for review, PTAL. Ignore the CI error, it's unrelated.

[jaimergp]

After the discussion in today's weekly:

Some background:

• I am concerned with adding more and more versions to the path settings. That's why I was pushing on "solving this once and for all". But that's just an opinion. If the team is ok with having several path versions, then my asks in items below are not so important.

My asks:

• Build IDs used to identify each attempt in the database shouldn't be so tied to the final path. Instead, we should resolve the lockfile first, and then compute the filesystem path from that output.
• Since that will involve more work, the current compromise of build-id + timestamp + username is ok. I do insist that this should just be an internal identifier in the database. The filesystem path could be computed later once input data is available.

As things stand, this is a fixed-length implementation of v2 right? We could try a hashed-lockfile in a future v4.

[nkaretnikov]

@jaimergp and I had a 15 min call about this.

To get this merged:

• [x] investigate the SQL query perf issue mentioned above
• [ ] add a $PREFIX/conda-meta/conda-store file
  • it must not have an extension to avoid collisions with other files in that directory
  • the format can be arbitrary JSON and should just provide a way for external (arbitrary) tools to map: <namespace>/<env> -> hash (if there're multiple generations, these should be stored in a sorted list by creation time)
  • this will provide a path forward for conda-store-ui and its Jupyter integration to use v3 while avoiding filesystem length issues
• update docs to mention this new file and its format.

[jaimergp]

I talked with @trallard and we agree that we are good to merge, with the following reservations:

• Mark v3 as experimental (i.e. call it v3_experimental) with no guarantees of hash stability across conda-store versions (for now).
• Document that this is going to receive iterations where we experiment with different ingredients for the hash. In the future we might want to revisit the lockfile hashing until we arrive to a more robust solution.
• Add a little section to the https://conda.store/community/policies/backwards-compatibility docs.

How does that sound? Thanks!

[nkaretnikov]

@jaimergp PTAL. Will rebase after this is approved.