search

MarcoIeni / release-plz

Publish Rust crates from CI with a Release PR.
https://release-plz.ieni.dev
Apache License 2.0
838 stars 85 forks source link

Inconsistent behaviour using release-plz with private, authenticated registries #1659

Closed mbodmer closed 1 month ago

mbodmer commented 2 months ago

Bug description

I am using cargo with a private registry (kellnr.io, sparse index) which is read and write protected.

Credentials are needed for every operation on it. Cargo takes them from the file ~/.cargo/credentials.toml which looks like:

# private registry, private crates
[registries.myregistry]
token = "myregistry-token-here"

# private registry, crates-io cache
[registries.myregistry-cratesio]
token = "myregistry-token-here"

If you want to read up more on configuration see: https://kellnr.io/documentation#configure-cargo and https://doc.rust-lang.org/cargo/reference/registry-authentication.html

When running release-plz release-pr --backend gitlab --git-token "mygit-token-here" --registry myregistry everything works fine:

  2024-09-06T07:33:04.355824Z  INFO  downloading packages from cargo registry
    in download with self=PackageDownloader { packages: ["mypackage"], directory: "/tmp/.tmpOkXTot", registry: Some("myregistry"), cargo_cwd: None }
    in next_versions
    in update
    in release_pr

note: Cloning into "/tmp/.tmpOkXTot"
    Updating `myregistry` index
  Credential cargo:wincred get myregistry
  Credential cargo:token get myregistry
note: Downloading mypackage 0.6.1
  2024-09-06T07:33:04.746360Z  INFO  mypackage: next version is 0.6.2
    in packages_to_update
    in next_versions
    in update
    in release_pr

  2024-09-06T07:33:06.022338Z  INFO  updated pr https://git.myhost.int/bla/mypackage/-/merge_requests/14
    in release_pr

however running release-plz release --backend gitlab --git-token "mygit-token-here" --registry myregistry does not work when there is an actual release waiting to be released:

  2024-09-06T09:15:57.652040Z  INFO  release-plz config file not found, using default configuration

  2024-09-06T09:15:57.652166Z DEBUG  initializing directory "/home/myuser/Projects/mypackage"
    in new

  2024-09-06T09:15:57.654658Z DEBUG  Processing: "git@git.myhost.int:bla/mypackage.git"

  2024-09-06T09:15:57.656102Z DEBUG  Scheme::SSH match for normalization

  2024-09-06T09:15:57.656119Z DEBUG  Normalizing ssh url: ["git@git.myhost.int", "bla/mypackage.git"]

  2024-09-06T09:15:57.656128Z DEBUG  Processing: "ssh://git@git.myhost.int/bla/mypackage.git"

  2024-09-06T09:15:57.656649Z DEBUG  The urlpath: "bla/mypackage.git"

  2024-09-06T09:15:57.656661Z DEBUG  rsplit results for metadata: ["mypackage.git", "bla"]

  2024-09-06T09:15:57.656751Z DEBUG  GitLab API URL: https://git.myhost.int/api/v4/projects/bla%2Fmypackage

  2024-09-06T09:15:57.656836Z DEBUG  manifest_dir: "/home/myuser/Projects/tests/mypackage"
    in release

  2024-09-06T09:15:57.657887Z DEBUG  project_root: "/home/myuser/Projects/tests/mypackage"
    in release

  2024-09-06T09:15:57.658060Z DEBUG  initializing directory "/home/myuser/Projects/tests/mypackage"
    in new
    in release

  2024-09-06T09:15:57.663288Z DEBUG  Release order: ["mypackage"]
    in release

  2024-09-06T09:15:57.670757Z DEBUG  starting new connection: https://crates.myregistry.int/
    in release

  2024-09-06T09:15:57.670913Z DEBUG  resolving host="crates.myregistry.int"

  2024-09-06T09:15:57.676870Z DEBUG  connecting to x.x.x.x:443
    in release

  2024-09-06T09:15:57.677254Z DEBUG  connected to x.x.x.x:443
    in release

  2024-09-06T09:15:57.679073Z DEBUG  binding client connection
    in release

  2024-09-06T09:15:57.679121Z DEBUG  client connection bound
    in release

  2024-09-06T09:15:57.679147Z DEBUG  send, frame: Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 }
    in release

  2024-09-06T09:15:57.679281Z DEBUG  pooling idle connection for ("https", crates.myregistry.int)
    in release

  2024-09-06T09:15:57.679380Z DEBUG  received, frame: Settings { flags: (0x0), max_concurrent_streams: 100 }
    in Connection with peer=Client
    in release

  2024-09-06T09:15:57.679405Z DEBUG  send, frame: Settings { flags: (0x1: ACK) }
    in Connection with peer=Client
    in release

  2024-09-06T09:15:57.679422Z DEBUG  received, frame: WindowUpdate { stream_id: StreamId(0), size_increment: 2147418112 }
    in Connection with peer=Client
    in release

  2024-09-06T09:15:57.679443Z DEBUG  send, frame: WindowUpdate { stream_id: StreamId(0), size_increment: 5177345 }
    in Connection with peer=Client
    in release

  2024-09-06T09:15:57.679462Z DEBUG  send, frame: Headers { stream_id: StreamId(1), flags: (0x5: END_HEADERS | END_STREAM) }
    in Connection with peer=Client
    in release

  2024-09-06T09:15:57.679815Z DEBUG  received, frame: Settings { flags: (0x1: ACK) }
    in Connection with peer=Client
    in release

  2024-09-06T09:15:57.679836Z DEBUG  received settings ACK; applying Settings { flags: (0x0), enable_push: 0, initial_window_size: 2097152, max_frame_size: 16384, max_header_list_size: 16384 }
    in Connection with peer=Client
    in release

  2024-09-06T09:15:57.680340Z DEBUG  received, frame: Headers { stream_id: StreamId(1), flags: (0x5: END_HEADERS | END_STREAM) }
    in Connection with peer=Client
    in release

  2024-09-06T09:15:57.680472Z DEBUG  send, frame: GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) }
    in Connection with peer=Client
    in release

  2024-09-06T09:15:57.680492Z DEBUG  Connection::poll; connection error, error: GoAway(b"", NO_ERROR, Library)
    in Connection with peer=Client
    in release

  2024-09-06T09:15:57.681045Z ERROR  can't determine if package is published

Caused by:
    0: timeout while publishing mypackage
    1: failed fetching sparse metadata
    2: the request was not authorized

Error: can't determine if package is published

Caused by:
    0: timeout while publishing mypackage
    1: failed fetching sparse metadata
    2: the request was not authorized

It only works when additionally given the registry token on the commandline like: release-plz release --backend gitlab --git-token "mygit-token-here" --registry myregistry --token "myregistry-token-here".

Log is more or less the same as above, but this time data is transferred and then cargo publish is called:

  2024-09-06T09:25:48.637309Z DEBUG  received, frame: Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) }
    in Connection with peer=Client
    in release

>  2024-09-06T09:25:48.637365Z DEBUG  received, frame: Data { stream_id: StreamId(1), flags: (0x1: END_STREAM) }
>    in Connection with peer=Client
>    in release

  2024-09-06T09:25:48.638026Z DEBUG  send, frame: GoAway { error_code: NO_ERROR, last_stream_id: StreamId(0) }
    in Connection with peer=Client
    in release

  2024-09-06T09:25:48.638061Z DEBUG  Connection::poll; connection error, error: GoAway(b"", NO_ERROR, Library)
    in Connection with peer=Client
    in release

>  2024-09-06T09:25:48.638063Z DEBUG  cargo publish --color always --manifest-path /home/myuser/Projects/tests/mypackage/Cargo.toml --package mypackage --registry myregistry --token my-registry-token
>    in release

  2024-09-06T09:25:48.810904Z DEBUG  cargo stderr:     Updating `myregistry` index
warning: manifest has no description, license, license-file, documentation, homepage or repository.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
   Packaging mypackage v0.6.3 (/home/myuser/Projects/tests/mypackage)
   Verifying mypackage v0.6.3 (/home/myuser/Projects/tests/mypackage)
   Compiling mypackage v0.6.3 (/home/myuser/Projects/tests/mypackage/target/package/mypackage-0.6.3)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.11s
    Packaged 10 files, 9.1KiB (3.2KiB compressed)
   Uploading mypackage v0.6.3 (/home/myuser/Projects/tests/mypackage)
    Uploaded mypackage v0.6.3 to registry `myregistry`
note: waiting for `mypackage v0.6.3` to be available at registry `myregistry`.
You may press ctrl-c to skip waiting; the crate should be available shortly.
   Published mypackage v0.6.3 at registry `myregistry`

Looks like in cargo.rs fn fetch_sparse_metadata() would only work with a token set for authenticated registries.

From afar, it looks like there is not much of common ground in usage between release-pr/update(release) in this regard. Maybe I did just not understand it yet.

Making available both, credentials.toml and the --token xy on the commandline somehow works in my case, but not when one of them is missing.

This does not make a lot of sense from a usage standpoint. Also this enforces credentials injection in multiple ways for release-plz.

Would work on a PR, but I don't understand the intentions of the design and also have not the cargo/http/auth knowledge for this.

MarcoIeni commented 2 months ago

I don't understand the intentions of the design

Consider what you don't understand or doesn't reasonate with you as "wrong design". Let me know how you would design this part so that we can reason about it together 👍

also have not the cargo/http/auth knowledge for this.

Yeah, every time I touch this part it feels like I need to study cargo auth from scratch 😂

davidBernardV8 commented 2 months ago

I had similar issue with a private artifactory. From my investigation, the cause is because the --token is used to access registry. But if --registry and --token are not provided no token is used (the value from environment is not used). This issue also prevent to use multiple registries and tokens.

I made a "fix", @MarcoIeni are you ok for a PR to discuss about it?

MarcoIeni commented 2 months ago

are you ok for a PR to discuss about it?

Sure. Go for it!!!

mbodmer commented 1 month ago

Cool, thank you! I'll try it out