search

beacon-biosignals / julia_pod

k8s native julia development
MIT License
10 stars 3 forks source link

keep docker layer cache from being invalidated when non-pinned deps change #37

Closed kolia closed 2 years ago

kolia commented 2 years ago

The problem as I understand it is that on update or adding of new dependencies, the non-pinned transitive dependencies of pinned deps can change versions, which invalidates the layer cache before the sysimage build. This makes re-launching julia_pod take a while.

A solution would require pinning all transitive dependencies of pinned deps before the docker build is started. However this would require adding all transitive dependencies of pinned repos as explicit dependencies (you can't pin something you don't depend on), which is a bit much.

We could save out a Manifest.toml and Project.toml that have all the transitive deps explicit and pinned in a subfolder like julia_pod_build, which gets used if it exists and gets created if it doesn't.

kolia commented 2 years ago

@haberdashPI who has been feeling this pain point as well.