✨ dagger modules - semver constraints and lock file

valorl commented 5 days ago

What are you trying to do?

Expanding on https://github.com/dagger/dagger/issues/6605 which addresses the case for a dagger update command.

However, currently dependencies in dagger.json work more as a lock file - any kind of dagger install will be stored as the resolved ref. This makes it a bit hard to reason about dependencies and update them safely, etc.

It would be nice to have a way to declaratively pin the dependencies to a human-friendly ref and/or ideally a semver expression. On install/update the constraints would be resolve to the actual latest ref.

To an extent, this is already the case, because dagger install can accept e.g. a branch/tag ref, but it's not declarative - the ref with which a dependency was initially installed is lost. So there would need to be two maps of dependencies, the resolved refs (~lockfile, current dependencies) and the "requirements" (branch/tag/semver expression, etc).

Usually the lock file is separate, which lets the user decide to not commit it and always fetch latest version (within constraints) in CI, at the cost of 100% reproducibility. IME this is kinda rare, so the lock file could also be part of dagger.json together with the requirements.

Assuming that setup, the commands might need to change, e.g.:

dagger install / dagger add: for adding a new requirement for a dependency (would also update lockfile with the latest compliant ref)
dagger upgrade: given the requirements, find the latest compliant refs and update the lockfile
(dagger update / dagger refresh / dagger restore / dagger fetch: for fetching the dependencies as per the lockfile, could also just be implicit and rely on running dagger develop or other commands)

Why is this important to you?

No response

How are you currently working around this?

No response

jedevc commented 17 hours ago

x-linking to https://github.com/dagger/dagger/issues/7898, which is gonna be resolved by https://github.com/dagger/dagger/pull/8587.

I think this is a step torwards dagger upgrade but not fully there.

Does the proposed changes to the syntax of dagger.json make sense to you? With source capturing the original from dagger install, and pin labelling the exact commit that we used.

valorl commented 16 hours ago

Yeah I believe that should cover it pretty (minus the ability to not commit the pin/lock versions, which I personally don't need). I like the idea of the combined source+pin string, it's probably simpler / better UX than having to know the difference between requirements and pinned dependencies, etc 👍

dagger / dagger