feat(storage): Store blob content-type in extended attributes

[jpetazzo]

After the discussion in #116, this stores the blob content types in extended attributes when using the filesystem backend.

If the underlying filesystem doesn't support extended attributes, storing blobs won't work; also, if extended attributes get removed, blobs won't be served anymore. We can relax this behavior if needed (i.e. log errors but still accept to store or serve blobs). However, since the Docker Engine (and possibly other container engines) won't accept to pull images from a registry that doesn't use correct content types for manifest files, it could be argued that it's better to give a hard fail. (Otherwise, the container engine gives cryptic error messages like "missing signature key".)

I can change that behavior (and log errors but still store/serve blobs to the filesystem) if you think it's better.

One last thing: I have very little knowledge of Go dependencies and Nix, so I hope I did the right thing here to add the dependency on pkg/xattr; but please double-check as it's my first time doing this :sweat_smile:

[jpetazzo]

Welp, the CI system doesn't support xattrs. :upside_down_face:

Should we try to change the CI system, or?

[flokli]

@jpetazzo I did some quick tinkering by manually spinning up a ubuntu:latest container on my machine:

❯ docker run --mount type=tmpfs,destination=/foo -it ubuntu:latest
root@152f82d17d81:/# apt update && apt install attr &> /dev/null
[…]
root@152f82d17d81:/# touch /foo/bar;setfattr -n "user.mime_type" -v "value" /foo/bar
setfattr: /foo/bar: Operation not supported
root@152f82d17d81:/# touch /root/bar;setfattr -n "user.mime_type" -v "value" /root/bar

Apparently, user xattrs on tmpfs aren't supported (see tmpfs(5)), and for other filesystems, the answer is "it depends" (xattr(7)).

I tinkered around with the CI a bit, and luckily the CI itself is backed by something supporting xattrs, it's just tmpfs doesn't support it.

I made https://github.com/google/nixery/commit/d2c007060c32882e12a2b0c044fd6dda7db6ab17.patch, which fixes CI, as can be seen in https://github.com/flokli/nixery/runs/2861162544.

Feel free to cherry-pick this on top.

[flokli]

I pushed that CI change to a PR at https://github.com/google/nixery/pull/128. If that is merged, this PR should become green after a rebase.

[tazjin]

Rebasing this PR on master should now resolve the CI issue, thanks to @flokli!

[flokli]

@jpetazzo can you press the rebase button?

[jpetazzo]

Thanks for the work on the CI! 👍🏻 I would have been happy to try and fix it, but I saw that there had been changes in the CI recently so I wasn't sure what was the current state. Looks like everything passes now, yay! 🎉

... And sorry for the slower turnaround on my end :)

[tazjin]

... And sorry for the slower turnaround on my end :)

I feel like I should write that sentence :sweat_smile: Thank you for the contribution!

I guess we don't need 7db252f36a68d875429a25e06d88fbfc804d84fd anymore after this and can probably revert it once merged?

[jpetazzo]

I guess we don't need 7db252f anymore after this and can probably revert it once merged?

Indeed! Would you like me to submit a PR that does both (i.e. with these commits + reverting the other one)?

[tazjin]

Feel free to send a separate CL reverting the old commit. Thanks for your contribution! :)