Open mark-rushakoff opened 1 year ago
I have confirmed that the .tar.zst approach is a valid workaround.
Listing the contents of the tar file looks like this:
$ tar tvf /tmp/myartifact.tar
-rw-r--r-- 0 0 0 2657758576 Feb 3 10:35 file.json
My Nomad artifact stanza looks like this (after I've separately zstandard-compressed the tarball and uploaded it somewhere):
artifact {
destination = "${NOMAD_TASK_DIR}/mydir/"
source = "https://example.com/myartifact.tar.zst"
}
And the resulting structure on the task's filesystem looks like this:
$ nomad alloc fs 52d1f mytask/local/mydir/
Mode Size Modified Time Name
-rw-r--r-- 2.5 GiB 2023-02-03T15:35:44Z file.json
Now that the file is world-readable, my main task can successfully read it.
I have a Nomad job containing an
artifact
stanza that downloads a single file from a URL, where the URL path is something like.../file.json.zstd
.The container for the Nomad task that needs to read this file, is running as a different user from the Nomad agent process.
go-getter
applies an unusual0622
permission on the extracted file; my container's user is not the file owner, so the user only has write permission:Allowing write access but not read access is unusual for regular files, which (almost?) every go-getter downloaded file would be.
I assume the intent was to set the permission as
0644
so that any user could read the extracted file.From what I see in the repo history, the
0622
setting was introduced in https://github.com/hashicorp/go-getter/pull/256. There is nothing in the PR's commit messages or the PR discussion mentioning why0622
was chosen.0622
appears to be used in all of bzip2, gzip, xz, and zstandard decompressors.Using
0644
may alleviate some of the issues people describe in https://github.com/hashicorp/nomad/issues/2625, but it won't address the root cause there.I will try to use a
.tar.zst
source as a workaround to retain world-readable permissions on my artifact file, but that does add an unusual step in my pipeline that generates and uploads the zstandard-compressed file today. If that doesn't work, I may be able to run a sidecar or prestart container in Nomad, with a root user who can chmod the file before my main container which runs as non-root.