conda-incubator / conda-store

Data science environments, for collaboration. ✨
https://conda.store
BSD 3-Clause "New" or "Revised" License
137 stars 44 forks source link

Rethink interaction of all conda-related tooling in conda-store #758

Open nkaretnikov opened 5 months ago

nkaretnikov commented 5 months ago

Context

@jaimergp suggested I open this for discussing long-term plans for conda-store with regard to how conda tools are used. Right now, we use conda as a library, via conda-lock, and via constructor. All these know nothing about each other (only implicitly, e.g., via a shared global conda cache), which might lead to problems. For example, https://github.com/conda-incubator/conda-store/pull/745#issuecomment-1916008070 was discovered because conda-store and conda-lock didn't communicate, so the shared cache was not used, which led to everything being downloaded on every build.

Value and/or benefit

Better performance and reliability.

Anything else?

No response