Open dhirschfeld opened 1 year ago
My first attempt failed with a "Could not upload" message:
❯ powerloader upload ./dist/boa/noarch/eq-utils-0.0.1-hcdf0012_0.conda:energy-quants/eq-utils:0.0.1-hcdf0012_0 -m oci://ghcr.io
[2023-02-20 21:17:11.912] [info] URL: https://ghcr.io/
Could not upload ./dist/boa/noarch/eq-utils-0.0.1-hcdf0012_0.conda:energy-quants/eq-utils:0.0.1-hcdf0012_0 to OCI Registry at https://ghcr.io/
Fortunately turning on verbose logging gave the clue:
❯ powerloader upload -vvv ./dist/boa/noarch/eq-utils-0.0.1-hcdf0012_0.conda:energy-quants/eq-utils:0.0.1 -m oci://ghcr.io
[2023-02-20 21:25:58.530] [info] URL: https://ghcr.io/
[2023-02-20 21:25:58.531] [warning] SSL verification is ENABLED
[2023-02-20 21:25:58.531] [warning] SSL verification is ENABLED
* Couldn't find host ghcr.io in the (nil) file; using defaults
* Trying 20.248.137.52:443...
* Connected to ghcr.io (20.248.137.52) port 443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
* CAfile: /opt/mambaforge/envs/quantdev/ssl/cacert.pem
* CApath: none
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN: server accepted h2
* Server certificate:
* subject: C=US; ST=California; L=San Francisco; O=GitHub, Inc.; CN=*.ghcr.io
* start date: Jul 19 00:00:00 2022 GMT
* expire date: Jul 19 23:59:59 2023 GMT
* subjectAltName: host "ghcr.io" matched cert's "ghcr.io"
* issuer: C=US; O=DigiCert Inc; CN=DigiCert TLS RSA SHA256 2020 CA1
* SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* h2h3 [:method: POST]
* h2h3 [:path: /v2/energy-quants/eq-utils/blobs/uploads/]
* h2h3 [:scheme: https]
* h2h3 [:authority: ghcr.io]
* h2h3 [accept: */*]
* Using Stream ID: 1 (easy handle 0x55d76d5ba1b0)
> POST /v2/energy-quants/eq-utils/blobs/uploads/ HTTP/2
Host: ghcr.io
accept: */*
* old SSL session ID is stale, removing
< HTTP/2 401
< content-type: application/json
< www-authenticate: Bearer realm="https://ghcr.io/token",service="ghcr.io",scope="repository:energy-quants/eq-utils:pull"
< date: Mon, 20 Feb 2023 11:25:59 GMT
< content-length: 73
< x-github-request-id: C8AA:41C9:0EFC:1433:63F358C7
<
* Connection #0 to host ghcr.io left intact
[2023-02-20 21:25:59.054] [error] Received 401: {"errors":[{"code":"UNAUTHORIZED","message":"authentication required"}]}
Could not upload ./dist/boa/noarch/eq-utils-0.0.1-hcdf0012_0.conda:energy-quants/eq-utils:0.0.1 to OCI Registry at https://ghcr.io/
So, for some reason it's failing authentication. I have an admin GitHub PAT in my GITHUB_TOKEN
environment variable which is picked up by the gh
CLI so the next reason was to figure out why it wasn't working for powerloader
.
Grepping the source, I found out that powerloader
expects the environment variable to be named GHA_PAT
. The name can be easily documented, however, it would be nice to also support the GITHUB_TOKEN
env var as does the gh
CLI:
❯ gh help environment GH_TOKEN, GITHUB_TOKEN (in order of precedence): an authentication token for github.com API requests. Setting this avoids being prompted to authenticate and takes precedence over previously stored credentials.
GH_ENTERPRISE_TOKEN, GITHUB_ENTERPRISE_TOKEN (in order of precedence): an authentication token for API requests to GitHub Enterprise. When setting this, also set GH_HOST.
Having configured the GHA_PAT
env var I still couldn't upload, getting the map::at
error I'd seen previously:
❯ powerloader upload -vvv ./dist/boa/noarch/eq-utils-0.0.1-hcdf0012_0.conda:energy-quants/eq-utils:0.0.1 -m oci://ghcr.io
[2023-02-20 22:04:21.503] [info] URL: https://ghcr.io/
[2023-02-20 22:04:21.505] [warning] SSL verification is ENABLED
[2023-02-20 22:04:21.505] [warning] SSL verification is ENABLED
[2023-02-20 22:04:21.505] [critical] Could not upload: map::at
i.e. I was attempting to incorporate the org/user name in the package name. It wasn't too difficult to figure out that powerloader
wanted the org/user name in the GHA_USER
env var.
After setting export GHA_USER='energy-quants'
the upload finally worked! :tada:
❯ powerloader upload -vvv eq-utils-0.0.1-hcdf0012_0.conda:eq-utils:0.0.1 -m oci://ghcr.io
<snip>
* Connection #0 to host ghcr.io left intact
[2023-02-20 22:08:17.888] [info] Uploaded 1 layers to energy-quants/eq-utils:0.0.1
Finished upload for eq-utils-0.0.1-hcdf0012_0.conda:eq-utils:0.0.1 to OCI Registry at https://ghcr.io/
export GHA_PAT=<github token>
export GHA_USER=<package owner> # org/user to publish to
powerloader upload <filepath to conda package>:<package name>:<package version> -m oci://ghcr.io
It seems a little strange to have the location the package is published to be controlled by hidden state (an env var).
It might be more obvious to specify that as a CLI argument - e.g.
powerloader upload eq-utils-0.0.1.conda:eq-utils:0.0.1 -m oci://ghcr.io --owner energy-quants
I guess that might not make sense for all OCI registries?
Anyway, it works - I've uploaded a conda
package as an OCI artifact to the GitHub Container Registry! That's awesome! :rocket::tada:
Figuring out how to install such a package will have to be a problem for another evening... 🤔
My first attempt at uploading a package failed:
...but I persevered (and grepped the source) and finally succeeded! :tada:
I'm just opening this issue to document the steps I had to take in case it is useful to others...