search

cue-lang / cue

The home of the CUE language! Validate and define text-based and dynamic configuration
https://cuelang.org
Apache License 2.0
5.02k stars 286 forks source link

internal/mod/modregistry: gitlab registry does not support modules artifact type #2817

Open rogpeppe opened 7 months ago

rogpeppe commented 7 months ago

What version of CUE are you using (cue version)?

$ cue version
v0.8.0-alpha.1

Does this issue reproduce with the latest stable release?

N/A (feature didn't exist in latest stable)

What did you do?

env TOKEN=xxxx
env GITLAB_USER=rogpeppe
env CUE_REGISTRY=registry.gitlab.com/rogpeppe/registrytest
env CUE_EXPERIMENT=modules
env DOCKER_CONFIG=$WORK
exec docker login registry.gitlab.com -u $GITLAB_USER --password $TOKEN
exec cue mod publish v1.0.0

-- cue.mod/module.cue --
module: "foo.com@v1"
-- m.cue --
package cuemodtest
x: 2

What did you expect to see?

A passing test

What did you see instead?

> env TOKEN=xxxx
> env GITLAB_USER=rogpeppe
> env CUE_REGISTRY=registry.gitlab.com/rogpeppe/registrytest
> env CUE_EXPERIMENT=modules
> env DOCKER_CONFIG=$WORK
> exec docker login registry.gitlab.com -u $GITLAB_USER --password $TOKEN
[stdout]
Login Succeeded
[stderr]
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in $WORK/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

> exec cue mod publish v1.0.0
[stderr]
cannot put module: cannot tag foo.com@v1.0.0: 400 Bad Request: manifest invalid: manifest invalid; detail: "unknown media type: application/vnd.cue.module.v1+json"
[exit status 1]
FAIL: /tmp/testscript4093019353/gitlab2.txtar/script.txtar:7: unexpected command failure

It turns out that gitlab does not support arbitrary media types. See https://gitlab.com/gitlab-org/container-registry/-/issues/973.

If the internal/modregistry code is changed to use a simple application/json type for the artifact type, the test above succeeds.

TODO

mxey commented 7 months ago

For the record, this does not apply to self-managed Gitlab installations, because they apparently do not use the new container registry (yet).

rogpeppe commented 6 months ago

These are the custom media types currently used by internal/mod/modregistry package:

The gitlab registry only has an issue with the former: it seems to ignore unusual blob media types.

Some documentation on how to construct a custom media type: https://datatracker.ietf.org/doc/html/rfc6838#section-3.2 More naming requirements here: https://datatracker.ietf.org/doc/html/rfc6838#section-4.2

The list of currently assigned IANA media types is here: https://www.iana.org/assignments/media-types/media-types.xhtml

Some instructions for registering a new media type are here: https://www.w3.org/2020/01/registering-mediatypes

Here is a previous Gitlab PR that added support for a new media type (application/vnd.ansible.collection): https://gitlab.com/gitlab-org/container-registry/-/merge_requests/1063

myitcv commented 6 months ago

Per our discussion offline, here is a summary of the proposed plan:

application/vnd.cue.module.v1+json
application/vnd.cue.modulefile.v1

In the worst case, we get pushback/feedback from IANA on our choice of media type. In which case we can revisit. Hence we should try and flush out issues there ASAP.

But the most important thing for GitLab users is to get things unblocked.

myitcv commented 6 months ago

But the most important thing for GitLab users is to get things unblocked.

@rogpeppe raised https://gitlab.com/gitlab-org/container-registry/-/issues/1203, which was then resolved by the now-merged https://gitlab.com/gitlab-org/container-registry/-/merge_requests/1588. We will update in this issue as we learn about the availability of that fix via GitLab "products".

rogpeppe commented 6 months ago

According to a reply to the above-mentioned issue, the fix should be released with GitLab 16.10, due out 2024-03-21.