Open Keno opened 3 years ago
However, when such a package is PkgCompile'd, PkgCompiler would go through, snapshot the referenced directory and turn these local artifacts into regular artifacts such that the rest of the PkgCompiler artifacts support just goes through.
An alternative is for the artifact loader (just like code loading) to understand non-content-addressed artifcacs and allow of a way to add/modify the way these are looked for.
Say we have an artifact.toml with
# Example Artifacts.toml file
[my_local_artifact]
path = "data"
The artifact string macro could expand artifact"my_local_artifact"
to something like:
macro artifact_str(name)
return quote
if !is_local_artifact(name)
# do the current stuff
end
local_artifact_dir = get(ENV, "LOCAL_ARTIFACT_DIR", nothing)
if local_artifact_dir === nothing
return joinpath(artifact_path, name)
else
# local artifact dir is overridden, namespace via pkgslug (same as code loading)
return joinpath(local_artifact_dir, pkgslug, name)
end
end
end
end
PackageCompiler would then just set LOCAL_ARTIFACT_DIR
and bundle all the local artifacts in a similar way to how the content addressed artifacts are bundled. That seems slightly less awkward than having to rewrite the Artifact.toml
file, make the package compile with that rewritten file etc.
When discussing #38682 on triage, the concern was brought up that while
@__DIR__
is useful, it is quite problematic for things like PkgCompiler that may end up distributing a compiled binary without the corresponding source tree, so packages relying on@__DIR__
to load configuration or test data will be unhappy. The best alternative mechanism we currently have is artifacts, which is fully integrated with PkgCompiler. However, it is a little high friction for the use case of referring to a few small files that are version controlled with the package. Thus, a proposal was made to add a variant of artifacts that is lower friction.In particular, we would add the ability to declare a folder in the package as a "local" artifact (by marking it appropriately in the Artifacts.toml). These local artifacts do not have a content hash, just a local relative path (and no download information, etc). They can then be referenced by name, just like any other artifact and during normal execution will just resolve the location in the local directory. However, when such a package is PkgCompile'd, PkgCompiler would go through, snapshot the referenced directory and turn these local artifacts into regular artifacts such that the rest of the PkgCompiler artifacts support just goes through.