JuliaLang / Pkg.jl

Pkg - Package manager for the Julia programming language
https://pkgdocs.julialang.org
Other
616 stars 258 forks source link

Allow passing a list of root manifests to Pkg.gc() #3717

Open ettersi opened 8 months ago

ettersi commented 8 months ago

We are using Julia in an embedded, offline system, and therefore when we make a release we have to bundle the Julia depot into the release so all the packages are available when we need them. For this purpose, it would be super useful if there was a way to convert a set of manifest files into a depot that is both lazy and pure. By this I mean:

Currently, I can almost achieve this objective using the following steps.

  1. Delete depot/logs to clear the list of "active" manifest files.
  2. Instantiate all the relevant projects.
  3. Run Pkg.gc(; collect_delay = Second(0)) to delete all the packages not used by the projects in Step 2.

The one catch with this solution is that Artifacts.toml files only get marked as used when artifacts are downloaded. This means that if I do the above on an empty depot, I get the result I want, but then when I run it again, I lose all the artifacts because they are already downloaded and so the artifact files don't get marked as used. It would be great if there was a way to make this work, and even better if I could do this using Pkg.gc(manifest_files).

ettersi commented 8 months ago

Replacing Step 2 with this seems to work:

using Pkg, Artifacts;
const world_age = Base.get_world_counter();
function Artifacts.load_artifacts_toml(artifacts_toml::String; kwargs...)
    Pkg.Types.write_env_usage(artifacts_toml, "artifact_usage.toml")
    Base.invoke_in_world(
        world_age,
        Artifacts.load_artifacts_toml,
        artifacts_toml;
        kwargs...
    )
end;
Pkg.instantiate()

But it's very hacky, so it would still be nice to have a better solution.