Support for Poetry

[di]

This project does not currently officially support Poetry (https://github.com/python-poetry/poetry).

The ability to support Poetry is currently blocked on https://github.com/python-poetry/poetry/pull/4086.

If/when that issue is resolved, we should revisit this issue and determine if Poetry is something we're able to officially support.

[jamesthompson]

Having poetry support would be ideal for us. Are you able to share how you tested this backend with the branch / PR here https://github.com/python-poetry/poetry/pull/4086? Any guidance you can give would be greatly appreciated.

[di]

@jamesthompson I installed Poetry from that users branch like so:

$ pip install git+https://github.com/Darsstar/poetry.git@pip-style-keyring-fallback

I wouldn't recommend this for production or even local development, as it's untrusted code that could be changed or go away at any time. But it does work!

[jamesthompson]

I'm assuming you have to set up the .pypirc an pip.conf files in the same manner in order to make it work. I'll give it a go tomorrow. Thanks for your rapid reply.

[di]

@jamesthompson No, you should follow Poetry's docs here instead: https://python-poetry.org/docs/repositories/#using-a-private-repository

[Algirdyz]

I worked around the issue by adding some scripts to my Dockerfile by adding: python -m keyring get url None It returns the password, but I can't get it to authenticate. I've tried adding it poetry config http-basic.name oauth2accessToken password. But it just gives a 401 error anyway. Is there something I'm missing with this way of authenticating?

[Bharathkumarraju]

@Algirdyz did you fix that error?

[di]

Update, for those following this thread: python-poetry/poetry#4086 was released in poetry==1.2.0a2, a prerelease.

[Bharathkumarraju]

@di thank you for the comment I am going to test it.

[jamesthompson]

@di I've just managed to build and install poetry=1.2.0a2 in order to try to test this out.

I'm doing the following:

adding the following dependencies to my pyproject.toml file:

[tool.poetry.dependencies]
python = "^3.8"
"keyrings.google-artifactregistry-auth" = "^0.0.3"
keyring = "^23.0.1"

running poetry install

running poetry config repositories.shared-pypi https://us-central1-python.pkg.dev/$TOPSECRETPROJECTNAMEHERE/our-pypi/ <- with the name of my repository being our-pypi.

Then running the gcloud auth login

then poetry publish --build -r our-pypi

and it's still prompting for the uername, so I added -u '' to silence that, but I get:

No suitable keyring backends were found and 404 Client Erro: Not found for url: https://us-central1-python.pkg.dev/$TOPSECRETPROJECTNAMEHERE/our-pypi <- replaced variables here obviously.

If you get chance would you be able to confirm if we need to set up any other files or environment variables to have the keyring library pick up the gcloud auth login creds it writes to the local environment.

Any basic guidance on the steps needed to publish to a GCP artifacts registry would be greatly appreciated.

[di]

Hi @jamesthompson, here's what I did to test this, starting from an empty directory:

$ pip install keyrings.google-artifactregistry-auth
Collecting keyrings.google-artifactregistry-auth
  Using cached keyrings.google_artifactregistry_auth-0.0.3-py3-none-any.whl (10.0 kB)
...
Installing collected packages: keyrings.google-artifactregistry-auth
Successfully installed keyrings.google-artifactregistry-auth-0.0.3

$ pip install poetry==1.2.0a2
Collecting poetry==1.2.0a2
  Using cached poetry-1.2.0a2-py3-none-any.whl (190 kB)
...
Installing collected packages: poetry
Successfully installed poetry-1.2.0a2

$ poetry init

This command will guide you through creating your pyproject.toml config.

Package name [my-sample-project]:
Version [0.1.0]:
Description []:
Author [Dustin Ingram <di@users.noreply.github.com>, n to skip]:
License []:
Compatible Python versions [^3.8]:

Would you like to define your main dependencies interactively? (yes/no) [yes] n
Would you like to define your development dependencies interactively? (yes/no) [yes] n
Generated file

[tool.poetry]
name = "my-sample-project"
version = "0.1.0"
description = ""
authors = ["Dustin Ingram <di@users.noreply.github.com>"]
readme = "README.md"
packages = [{include = "my_sample_project"}]

[tool.poetry.dependencies]
python = "^3.8"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Do you confirm generation? (yes/no) [yes]

$ cat pyproject.toml
[tool.poetry]
name = "my-sample-project"
version = "0.1.0"
description = ""
authors = ["Dustin Ingram <di@users.noreply.github.com>"]
readme = "README.md"
packages = [{include = "my_sample_project"}]

[tool.poetry.dependencies]
python = "^3.8"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

$ poetry config repositories.google https://us-central1-python.pkg.dev/di-catchall/quickstart-pypi-repo

$ mkdir my_sample_project

$ touch my_sample_project/__init__.py

$ touch README.md

$ poetry publish --build -r google
Building my-sample-project (0.1.0)
  - Building sdist
  - Built my-sample-project-0.1.0.tar.gz
  - Building wheel
  - Built my_sample_project-0.1.0-py3-none-any.whl

Publishing my-sample-project (0.1.0) to google
 - Uploading my-sample-project-0.1.0.tar.gz 100%
 - Uploading my_sample_project-0.1.0-py3-none-any.whl 100%

And then to install the package as a dependency of another project:

$ echo "
[[tool.poetry.source]]
name = \"google\"
url = \"https://us-central1-python.pkg.dev/di-catchall/quickstart-pypi-repo/simple\"" >> pyproject.toml

$ cat pyproject.toml
[tool.poetry]
name = "some-other-project"
version = "0.1.0"
description = ""
authors = ["Dustin Ingram <di@users.noreply.github.com>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.8"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

[[tool.poetry.source]]
name = "google"
url = "https://us-central1-python.pkg.dev/di-catchall/quickstart-pypi-repo/simple"

$ poetry add my-sample-project
Using version ^0.1.0 for my-sample-project

Updating dependencies
Resolving dependencies... (0.2s)

Writing lock file

Package operations: 1 install, 0 updates, 0 removals

  • Installing my-sample-project (0.1.0)

No other files or configuration should be necessary.

[Bharathkumarraju]

@jamesthompson make sure you authenticate to google cloud first. It worked for me as well. check my testing in this repo: https://github.com/Bharathkumarraju/poetry-gcp-pypi

[jamesthompson]

@Bharathkumarraju Yeah no I got this to work fine as well. Thanks a lot @di for the quick help, it's really appreciated.

[TaridaGeorge]

I cannot get it to work. Do I have to export GOOGLE_APPLICATION_CREDENTIALS or do gcloud auth login? I've tried both of them but with no luck.

poetry --version Poetry (version 1.2.0a2)

[Bharathkumarraju]

@TaridaGeorge you can try to run this gcloud auth application-default login

[Kreshu]

@TaridaGeorge how did you install poetry? With pip or installation script? If the latter one - poetry will be installed in it's own venv and you have to add GCP keyring in there with poetry plugin add keyrings.google-artifactregistry-auth

[TaridaGeorge]

I had it installed via pipx and I taught that this could be the problem. I've installed the keyrings.google-artifactregistry-auth package into the pipx created venv for the poetry app but with no results. I've then uninstalled the poetry from the pipx and I've installed it according to the github page:

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python -
poetry self update --preview
poetry --version  => Poetry (version 1.2.0a2)
gcloud auth application-default login
poetry plugin add keyrings.google-artifactregistry-auth
poetry config repositories.my-private-google-repo https://europe-python.pkg.dev/project_id/repo-name/simple
poetry publish -r my-private-google-repo

Is not working.

poetry publish -r my-private-google-repo

  DBusErrorResponse

  [org.freedesktop.Secret.Error.NoSuchObject] ("The '/org/freedesktop/secrets/collection/login' object does not exist",)

  at ~/.local/share/pypoetry/venv/lib/python3.9/site-packages/jeepney/wrappers.py:214 in unwrap_msg
      210│     This is to be used with replies to method_call messages, which may be
      211│     method_return or error.
      212│     """
      213│     if msg.header.message_type == MessageType.error:
    → 214│         raise DBusErrorResponse(msg)
      215│ 
      216│     return msg.body
      217│ 

The following error occurred when trying to handle this error:

  ItemNotFoundException

  Item does not exist!

  at ~/.local/share/pypoetry/venv/lib/python3.9/site-packages/secretstorage/util.py:49 in send_and_get_reply
       45│              try:
       46│                      return self._connection.send_and_get_reply(msg, unwrap=True)
       47│              except DBusErrorResponse as resp:
       48│                      if resp.name in (DBUS_UNKNOWN_METHOD, DBUS_NO_SUCH_OBJECT):
    →  49│                              raise ItemNotFoundException('Item does not exist!') from resp
       50│                      elif resp.name in (DBUS_SERVICE_UNKNOWN, DBUS_EXEC_FAILED,
       51│                                         DBUS_NO_REPLY):
       52│                              data = resp.data
       53│                              if isinstance(data, tuple):

The following error occurred when trying to handle this error:

  PromptDismissedException

  Prompt dismissed.

  at ~/.local/share/pypoetry/venv/lib/python3.9/site-packages/secretstorage/collection.py:145 in create_collection
      141│      if len(collection_path) > 1:
      142│              return Collection(connection, collection_path, session=session)
      143│      dismissed, result = exec_prompt(connection, prompt)
      144│      if dismissed:
    → 145│              raise PromptDismissedException('Prompt dismissed.')
      146│      signature, collection_path = result
      147│      assert signature == 'o'
      148│      return Collection(connection, collection_path, session=session)
      149│ 

The following error occurred when trying to handle this error:

  InitError

  Failed to create the collection: Prompt dismissed..

  at ~/.local/share/pypoetry/venv/lib/python3.9/site-packages/keyring/backends/SecretService.py:63 in get_preferred_collection
       59│                 collection = secretstorage.Collection(bus, self.preferred_collection)
       60│             else:
       61│                 collection = secretstorage.get_default_collection(bus)
       62│         except exceptions.SecretStorageException as e:
    →  63│             raise InitError("Failed to create the collection: %s." % e)
       64│         if collection.is_locked():
       65│             collection.unlock()
       66│             if collection.is_locked():  # User dismissed the prompt
       67│                 raise KeyringLocked("Failed to unlock the collection!")

The following error occurred when trying to handle this error:

  KeyRingError

  Unable to retrieve the password for poetry-repository-my-private-google-repo from the key ring

  at ~/.local/share/pypoetry/venv/lib/python3.9/site-packages/poetry/utils/password_manager.py:46 in get_password
       42│ 
       43│         try:
       44│             return keyring.get_password(name, username)
       45│         except (RuntimeError, keyring.errors.KeyringError):
    →  46│             raise KeyRingError(
       47│                 f"Unable to retrieve the password for {name} from the key ring"
       48│             )
       49│ 
       50│     def set_password(self, name: str, username: str, password: str) -> None:

If I try to publish my package via twine with the twine upload --repository-url https://europe-python.pkg.dev/project_id/repo-name/ dist/* --verbose it works.

[MadJlzz]

@TaridaGeorge I had exactly the same problem. Even by doing what @Kreshu mentionned with:

poetry plugin add keyrings.google-artifactregistry-auth

Didn't solve the problem. To make it work, I've used pip to install poetry:

pip install poetry==1.2.0a2
pip install keyrings.google-artifactregistry-auth
gcloud auth application-default login
poetry config repositories.google https://europe-python.pkg.dev/project_id/repo-name
poetry publish -r google 

You don't put /simple/ at the end for repositories.X as this url is used only for deploying, not retrieving.

Hope that helps.

[jeethridge]

@TaridaGeorge I had exactly the same problem. Even by doing what @Kreshu mentionned with:

poetry plugin add keyrings.google-artifactregistry-auth

Didn't solve the problem. To make it work, I've used pip to install poetry:

pip install poetry==1.2.0a2
pip install keyrings.google-artifactregistry-auth
gcloud auth application-default login
poetry config repositories.google https://europe-python.pkg.dev/project_id/repo-name
poetry publish -r google 

You don't put /simple/ at the end for repositories.X as this url is used only for deploying, not retrieving.

Hope that helps.

This is exactly the approach that worked for me. After a fair amount of struggling trying to get it work with poetry installed from the script I finally removed it manually (on OSX with zsh+oh-my-zsh) by deleting /Users/<username>/.local/bin/poetry and removing the POETRY_* environment variables I had in my .zprofile. After I got rid of the global poetry install and went the pip route everything worked like a charm. This seems like the only way to get the keyrings.google-artifactregistry-auth plugin to work in harmony with poetry as using the poetry plugin commands to install it gave me errors as well. FWIW I was running python 3.9.5 using pyenv when I had success with the above procedure.

[MadJlzz]

I tried again on my side and saw something really interesting. Normally, when running:

poetry plugin add keyrings.google-artifactregistry-auth

poetry should install "keyrings.google-artifactregistry-auth" dependency in poetry's venv. After a quick check, it is indeed installed.

But, from the documentation, keyring --list-backends output should now show the new backend but it doesn't:

keyring.backends.Windows.WinVaultKeyring (priority: 5)
keyring.backends.fail.Keyring (priority: 0)
keyring.backends.chainer.ChainerBackend (priority: -1)

Something is happening with poetry plugin add that doesn't let keyrings.google-artifactregistry-auth register as a keyring backend. 🤔

I tried then to install the package from poetry's venv with pip:

pip install keyrings.google-artifactregistry-auth

And checked the output again:

keyring.backends.chainer.ChainerBackend (priority: 10)
keyrings.gauth.GooglePythonAuth (priority: 9)
keyring.backends.fail.Keyring (priority: 0)
keyring.backends.Windows.WinVaultKeyring (priority: 5)

I then simply configured poetry's to use another registry for publishing:

poetry config repositories.google https://europe-python.pkg.dev/project_id/repo-name
poetry build
poetry publish -r google 

and it worked as expected!

Also, for retrieving, you'll need to add:

[[tool.poetry.source]]
name = "google"
url = "https://europe-python.pkg.dev/project_id/repo-name/simple/"

and sadly, because poetry creates a new venv for you, you'll need to add "keyrings.google-artifactregistry-auth" as a dependency. (i am putting it as a dev dependency for now)

[NixBiks]

Just to be clear; it is not possible to use google python artifact registry with poetry 1.1.12? It is only possible with bleeding edge poetry (which is not recommended in production AFAIK)?

[MadJlzz]

@mr-bjerre To me yes, poetry 1.1.12 is not able to handle correctly secrets from Google keyring backend. See https://github.com/python-poetry/poetry/pull/4086

[MadJlzz]

For people like me that tried to make this work, I finally have found a very reproducible way that you could propagate to your teams in case you really need the keyring to work properly,

Personally I am using pyenv and when testing, I was starting from a fresh environment:

pip freeze > unins ; pip uninstall -y -r unins ; del unins

Install poetry directly with pip from the master branch of poetry's repository:

pip install git+https://github.com/python-poetry/poetry.git@master

Installing from the master branch is a MUST if you are working on Windows as the last pre-release version is missing a crucial fix: https://github.com/python-poetry/poetry/pull/4549

Install the Google keyring helper:

pip install keyrings.google-artifactregistry-auth

If you're using pyenv, rehash your binaries:

pyenv rehash

Finally, log in with your user credentials (you will require the correct permissions in GCP for pull/push packages)

gcloud auth login

From there, go over your pyproject.toml and add a block like:

[[tool.poetry.source]]
name = "private-pypi"
url = "https://europe-python.pkg.dev/<PROJECT_ID>/<REPO_NAME>/simple/"
secondary = true

You should be able to fetch private package from this registry.

To publish packages, simply configure poetry like so:

poetry config repositories.private-pypi https://europe-python.pkg.dev/<PROJECT_ID>/<REPO_NAME>/

and then publish with

poetry publish -r private-pypi --build

I hope this helps.

[Vichoko]

Thanks a lot @MadJlzz for the replicate. I had been having trouble with poetry catching up with the gcloud's keyring and it seems that performing gcloud auth login after installing the keyring is the key to making it work for both installing and uploading wth poetry.

[MadJlzz]

@Vichoko I started working on a plugin for poetry that would install automatically keyrings.google-artifactregistry-auth inside the venv of the main poetry cmd: https://github.com/MadJlzz/poetry-artifactregistry-plugin

I made it work with the last commit present on poetry repository:

curl -sSL https://install.python-poetry.org | python3 - --git https://github.com/python-poetry/poetry.git@master

and

poetry plugin add poetry-artifactregistry-plugin

[japerry911]

@MadJlzz , you are a hero, thank you.

Quick question:

I have to run this command to get it to install my private artifact repo package, otherwise it simply looks in PyPi public and fails with 404 Not Found. Am I potentially doing something wrong?

Command: poetry add --source <name> <package-name>

pyproject.toml section that I added:

[[tool.poetry.source]]
name = <name>
url = <artifact-repo-url>

I tried default=false and secondary=true combo with no luck.

Thanks again for your posts, very helpful sir.

[MadJlzz]

@japerry911 Here's what I have on my side:

[[tool.poetry.source]]
name = "<name>"
url = "https://europe-python.pkg.dev/<google-project-id>/ngt-python/simple/"
default = false
secondary = true

The /simple/ is important to mention at the end.

[japerry911]

Thanks for your help!

I was able to get it to work in Cloud Build woohoo!

[Vichoko]

Is there any estimated time for the first stable Poetry 1.2 release?

[hvpeteet]

Not sure if this will help anyone but to get this to work with poetry installed from the installer (instead of pip), you can run poetry self add "keyrings.google-artifactregistry-auth==1.0.0" (instead of pip installing it) since poetry won't pick up the pip install from https://github.com/GoogleCloudPlatform/artifact-registry-python-tools/issues/17#issuecomment-1040309388.

[MadJlzz]

Not sure if this will help anyone but to get this to work with poetry installed from the installer (instead of pip), you can run poetry self add "keyrings.google-artifactregistry-auth==1.0.0" (instead of pip installing it) since poetry won't pick up the pip install from #17 (comment).

This is the way to go indeed when 1.2.0 will be released!

To rephrase, poetry self add will install the given package in the venv of poetry's installation :)

[vanyakosmos]

You can also install preview version using this command:

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python - -p

Also side note for someone using docker: don't forget to update lock file, adding packages by hands into pyproject.toml wont do.

[adamwojt]

What worked for me locally (combination of things from comments above):

1. Using latest beta for poetry pip install poetry==1.2.0b3
2. Installing keyrings pip3 install keyrings.google-artifactregistry-auth
3. Having gcloud setup or exporting GOOGLE_APPLICATION_CREDENTIALS accordingly.
4. Adding new repo in pyproject.toml

   [[tool.poetry.source]]
   name = "company-common-repo"
   url = "https://europe-python.pkg.dev/my_project_id/company-common-repo/simple"
   secondary = true
   default = false

5. Adding dependency like this allows pulling:

   company-common-package = {version = "0.1.0", source = "company-common-repo"}

Additional For publishing:

1. poetry config repositories.company-common-repo https://europe-python.pkg.dev/my_project_id/company-common-repo/
2. poetry publish -r company-common-repo --build

For installing packages from private repo in Dockerfile:

FROM python:3.9.12

WORKDIR /app

RUN pip3 install --upgrade pip
RUN pip install poetry==1.2.0b3
RUN pip3 install keyrings.google-artifactregistry-auth

COPY pyproject.toml poetry.lock ./

RUN poetry config virtualenvs.create false
RUN --mount=type=secret,id=google_creds,target=/app/google_creds.json export GOOGLE_APPLICATION_CREDENTIALS="/app/google_creds.json" && poetry install --no-dev --no-interaction --no-ansi --no-root

COPY ./src/ /app/src/

# Install root package now, it will help caching when building locally.
RUN poetry install --no-dev --no-interaction --no-ansi

USER 1001
ENTRYPOINT ["python3"]
CMD ["-m", "company_common"]

To build with docker:

1. export DOCKER_BUILDKIT=1
2. docker build --secret "id=google_creds,src=$GOOGLE_APPLICATION_CREDENTIALS" -t my_tag .

Gitlab pipeline for publishing package to artifacts on pushing tag.

#!/bin/bash
# To be run from gitlab CI
set -euo pipefail

pip install poetry==1.2.0b3

CURRENT_PACKAGE_VERSION=`poetry version -s`
echo "Package version in poetry: $CURRENT_PACKAGE_VERSION"
TAG=$CI_COMMIT_TAG

if [[ "v$CURRENT_PACKAGE_VERSION" != "$TAG" ]]; then
  echo "Tag and Poetry package version mismatch, please check both."
  exit 1
fi

# Build and push
pip3 install keyrings.google-artifactregistry-auth
poetry config repositories.company-common-repo https://europe-python.pkg.dev/my_project_id/company-common-repo/
poetry publish -r company-common-repo --build

echo "Done."
echo "https://console.cloud.google.com/artifacts/python/my_project_id/europe/company-common/company-common-package/$CURRENT_PACKAGE_VERSION?project=my_project_id"

[ESKYoung]

To flag, Poetry 1.2 has now been released.

However, with the latest version of keyrings.google-artifactregistry-auth (1.1.0), I can't authenticate with Google Artifact Registry. If I downgrade to 1.0.0 everything works as expected. Steps to reproduce:

# Part of pyproject.toml
[[tool.poetry.source]]
name = "<name>"
url = "https://europe-python.pkg.dev/<google-project-id>/ngt-python/simple/"
default = false
secondary = true

poetry self add "keyrings.google-artifactregistry-auth"
poetry add <package_name>

Error message:

Source (<name>): Authorization error accessing https://europe-python.pkg.dev/<google-project-id>/ngt-python/simple/<package_name>/

Could not find a matching version of package <package_name>

If you replace the above poetry self add step with:

poetry self add "keyrings.google-artifactregistry-auth==1.0.0"

everything seems to work fine.

[megan-kuo]

Sorry about that, I think the issue was due to a dependency from the keyring library.

Could you try with the latest version of keyrings.google-artifactregistry-auth (1.1.1)? GoogleCloudPlatform/artifact-registry-python-tools/#35 should have fixed the issue.

Thanks!

[ESKYoung]

Thanks for the rapid response @megan-kuo!! Seems to have fixed it!

[MadJlzz]

Seems that the support is done then! Should we close this @di ?

[ESKYoung]

@MadJlzz - should clarify, I've not tested publishing a package to Google Artifact Registry with Poetry 1.2, only reading from it!

Might be a few days/weeks before I get to it (I last checked with Poetry 1.2.0b2, and it was working).

[di]

I don't work on this project anymore, I'll defer to @megan-kuo!

[MadJlzz]

I don't work on this project anymore, I'll defer to @megan-kuo!

Sure thing!

@MadJlzz - should clarify, I've not tested publishing a package to Google Artifact Registry with Poetry 1.2, only reading from it!

Might be a few days/weeks before I get to it (I last checked with Poetry 1.2.0b2, and it was working).

@ESKYoung, I just tried on Windows 11 with Poetry 1.2.0 and keyrings.google-artifactregistry-auth 1.1.1 and it works like a charm.

poetry source add demo https://europe-west1-python.pkg.dev/madsanbox/poetry-120-demo/simple/ # this is used for pulling packages
poetry config repositories.demo https://europe-west1-python.pkg.dev/madsanbox/poetry-120-demo/  # this is used for pushing packages

If it helps, I can provide a working example as a kickstart.

[ESKYoung]

@MadJlzz sounds good! I've confirmed package publication works on macOS as well!

[MadJlzz]

@megan-kuo I think we can close this one

[megan-kuo]

will do, thanks!

[LeonB28]

still facing with 1.2.2: 401 Client Error: Unauthorized for url: https://us-central1-python.pkg.dev/repo/quickstart-python-repo/
while I am able to upload using twine

[Finnepinnen]

Works with version 1.2.0 and 1.2.2 but not with version 1.3.0

[megan-kuo]

it looks like we have issue #36 open tracking this issue.

I'm not very familiar with poetry but if you would be willing to send over a pull request I'd be happy to take a look. thanks!

[Matesanz]

Such a great contribution @adamwojt (https://github.com/GoogleCloudPlatform/artifact-registry-python-tools/issues/17#issuecomment-1199289447) Thank you so much. :tada:

:question: Just a few questions regarding:

3, Having gcloud setup or exporting GOOGLE_APPLICATION_CREDENTIALS accordingly.

:point_right: Is it a service account credentials json? :point_right: Is it possible to use a user application_default_credentials.json? (the one stored by default at $HOME/.config/gcloud/application_default_credentials.json when running gcloud auth application-default)

The point is to allow access by user to the artifactory and not creating a single service account for everybody as it carries certain security risks.

[adamwojt]

@Matesanz Thanks! It can be both, service account or your default credentials. As long as it has access to artifacts :) We were giving read rights for artifacts repository and issuing it to developers. CI had service account with read/write that enabled it to publish :)

[Vichoko]

So we have migrated all our private dependencies from git to the artifact registry. And most of the team has it working already. But we have some people having lots of issues installing the python, poetry & gcloud auth correctly.

I've personally assisted these people and mimicked the exact steps but it seems that the keyring is failing to retrieve gcloud auth login credentials.

My steps are the following:

1. Remove python (pyenv, conda, etc)(
2. Install python 3.8 or 3.10 with homebrew (We use MacOS)
3. Add python3.8 or 3.10 as a python alias (same for pip3.8/3.10 respectively)
4. Install latest poetry with install script.
5. Install keyring with poetry self add keyrings.google-artifactregistry-auth -n
6. Run gcloud auth login
7. Rungcloud auth application-default login
8. Create venv with python -m venv .venv
9. Activate venv with .venv/bin/activate
10. Run poetry install

This fails with a bunch of 401 Unauthorized errors. The user has all permissions on the artifact registry. I feel the keyring isn't retrieving gcloud credentials, because if I run the following code:

import keyring

# Get the Google Cloud credentials from the keyring
credentials = keyring.get_password("google-artifactregistry-auth", "default")

if credentials:
    print("Credentials found in the keyring.")
    # Do something with the credentials, such as decoding them and checking their contents
else:
    print("Credentials not found in the keyring.")

It prints: Credentials not found in the keyring.

[bitnahian]

Unable to publish to a private GCR without gcloud cli installed in my image. Have exported the correct GOOGLE_APPLICATION_CREDENTIALS and GOOGLE_CLOUD_PROJECT environment variables, but still receiving a 405. Can confirm this in a fresh Dev Container environment without gcloud cli installed.

Steps to reproduce:

 export GOOGLE_APPLICATION_CREDENTIALS=<PATH TO CREDENTIALS> 
 export GOOGLE_CLOUD_PROJECT=<PROJECT>

 poetry self add "keyrings.google-artifactregistry-auth==1.1.2"
 poetry config repositories.<REPO> https://us-docker.pkg.dev/<PROJECT>/<REPO>
 poetry publish -r my_package --build 

Error

HTTP Error 405: Method Not Allowed | b'\n<!DOCTYPE html>\n<html lang=en>\n  <meta charset=utf-8>\n  <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">\n  <title>Error 405 (Method Not Allowed)!!1\n  <style>\n    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}\n  \n  <a href=//www.google.com/><span id=logo aria-label=Google>\n  <p>405. <ins>That\xe2\x80\x99s an error.\n  <p>  <ins>That\xe2\x80\x99s all we know.\n'

This works fine in my local environment where I have gcloud configured.

[deschman]

Would it be possible for the solution in this comment to be documented somewhere in the official docs? Figuring this out became quite the rabbit trail.