GoogleCloudPlatform / cloud-sql-python-connector

A Python library for connecting securely to your Cloud SQL instances.
Apache License 2.0
278 stars 65 forks source link

system.test_asyncpg_iam_auth: test_connection_with_asyncpg_iam_auth failed #842

Closed flaky-bot[bot] closed 11 months ago

flaky-bot[bot] commented 11 months ago

Note: #694 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.


commit: 71b9347e265398ae2c3a0db4c1a7595ac199b9bf buildURL: https://github.com/GoogleCloudPlatform/cloud-sql-python-connector/actions/runs/6177038374 status: failed

Test output
@pytest.fixture(name="pool")
    async def setup() -> AsyncGenerator:
        pool = await init_connection_pool()
>       async with pool.connect() as conn:

tests/system/test_asyncpg_iam_auth.py:59: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/ext/asyncio/base.py:127: in __aenter__
    return await self.start(is_ctxmanager=True)  # type: ignore
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/ext/asyncio/engine.py:268: in start
    await greenlet_spawn(self.sync_engine.connect)
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py:190: in greenlet_spawn
    result = context.throw(*sys.exc_info())
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/engine/base.py:3264: in connect
    return self._connection_cls(self)
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/engine/base.py:145: in __init__
    self._dbapi_connection = engine.raw_connection()
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/engine/base.py:3288: in raw_connection
    return self.pool.connect()
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/pool/base.py:452: in connect
    return _ConnectionFairy._checkout(self)
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/pool/base.py:1267: in _checkout
    fairy = _ConnectionRecord.checkout(pool)
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/pool/base.py:716: in checkout
    rec = pool._do_get()
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/pool/impl.py:170: in _do_get
    self._dec_overflow()
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py:147: in __exit__
    raise exc_value.with_traceback(exc_tb)
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/pool/impl.py:167: in _do_get
    return self._create_connection()
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/pool/base.py:393: in _create_connection
    return _ConnectionRecord(self)
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/pool/base.py:678: in __init__
    self.__connect()
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/pool/base.py:903: in __connect
    pool.logger.debug("Error on connect(): %s", e)
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py:147: in __exit__
    raise exc_value.with_traceback(exc_tb)
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/pool/base.py:898: in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/pool/base.py:365: in 
    return lambda rec: creator_fn()
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/ext/asyncio/engine.py:112: in creator
    return sync_engine.dialect.dbapi.connect(  # type: ignore
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py:917: in connect
    await_only(creator_fn(*arg, **kw)),
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py:125: in await_only
    return current.driver.switch(awaitable)  # type: ignore[no-any-return]
.nox/system-3-8/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py:185: in greenlet_spawn
    value = await result
tests/system/test_asyncpg_iam_auth.py:38: in getconn
    conn: asyncpg.Connection = await connector.connect_async(
google/cloud/sql/connector/connector.py:208: in connect_async
    instance = Instance(
google/cloud/sql/connector/instance.py:250: in __init__
    self._credentials = _auth_init(credentials)
google/cloud/sql/connector/utils.py:124: in _auth_init
    credentials, _ = default(scopes=scopes)
.nox/system-3-8/lib/python3.8/site-packages/google/auth/_default.py:675: in default
    project_id = credentials.get_project_id(request=request)
.nox/system-3-8/lib/python3.8/site-packages/google/auth/external_account.py:339: in get_project_id
    self.before_request(request, "GET", url, headers)
.nox/system-3-8/lib/python3.8/site-packages/google/auth/credentials.py:151: in before_request
    self.refresh(request)
.nox/system-3-8/lib/python3.8/site-packages/google/auth/external_account.py:360: in refresh
    self._impersonated_credentials.refresh(request)
.nox/system-3-8/lib/python3.8/site-packages/google/auth/impersonated_credentials.py:247: in refresh
    self._update_token(request)
.nox/system-3-8/lib/python3.8/site-packages/google/auth/impersonated_credentials.py:260: in _update_token
    self._source_credentials.refresh(request)
.nox/system-3-8/lib/python3.8/site-packages/google/auth/external_account.py:378: in refresh
    subject_token=self.retrieve_subject_token(request),
.nox/system-3-8/lib/python3.8/site-packages/google/auth/identity_pool.py:158: in retrieve_subject_token
    self._get_token_data(request),
.nox/system-3-8/lib/python3.8/site-packages/google/auth/identity_pool.py:167: in _get_token_data
    return self._get_url_data(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = 
request = 
url = 'https://pipelines.actions.githubusercontent.com/umAmnh0OhcfbtGEt7J16Yga6HsgM8dYIhPxbPiOYFLVwMnfbKz/00000000-0000-0000...cations%2Fglobal%2FworkloadIdentityPools%2Fgh-13a715-cloud-sql-pyt-dd1c5f%2Fproviders%2Fgh-13a715-cloud-sql-pyt-dd1c5f'
headers = {'Authorization': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6ImVCWl9jbjNzWFlBZDBjaDRUSEJLSElnT3dPRSJ9.eyJuYW1...1HxZ_l2_o3SbuMMgb4fJBwwOJSH8KrPldkjmVftWRZ9HzzyLdZrU_CqcQZeyr5S51wqzainKFLPUKb21Wy7DsC7J1l54y5oWqFci6nIRlQCS-HZigU17Q'}

    def _get_url_data(self, request, url, headers):
        response = request(url=url, method="GET", headers=headers)

        # support both string and bytes type response.data
        response_body = (
            response.data.decode("utf-8")
            if hasattr(response.data, "decode")
            else response.data
        )

        if response.status != 200:
>           raise exceptions.RefreshError(
                "Unable to retrieve Identity Pool subject token", response_body
            )
E           google.auth.exceptions.RefreshError: ('Unable to retrieve Identity Pool subject token', '{ "message": "GitHub Actions is temporarily unavailable. Please visit https://www.githubstatus.com/ for the status of our services.", "ref": "Ref A: A2FB2A6DA4E04287B3BA325795C34719 Ref B: SN1EDGE1306 Ref C: 2023-09-13T19:36:44Z" }')

.nox/system-3-8/lib/python3.8/site-packages/google/auth/identity_pool.py:189: RefreshError
flaky-bot[bot] commented 11 months ago

Looks like this issue is flaky. :worried:

I'm going to leave this open and stop commenting.

A human should fix and close this.


When run at the same commit (71b9347e265398ae2c3a0db4c1a7595ac199b9bf), this test passed in one build (https://github.com/GoogleCloudPlatform/cloud-sql-python-connector/actions/runs/6177038374) and failed in another build (https://github.com/GoogleCloudPlatform/cloud-sql-python-connector/actions/runs/6177038374).

jackwotherspoon commented 11 months ago

Github Actions Auth issue