fsspec / s3fs

S3 Filesystem
http://s3fs.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
862 stars 271 forks source link

"coroutine object is not subscriptable" error calling get_delegated_s3pars #774

Open lamorgiac opened 1 year ago

lamorgiac commented 1 year ago

I'm trying to connect to a MinIO S3 compatible storage and to create a session token, with this code:

            fs = S3FileSystem(key='minioadmin', secret='minioadmin',
                              endpoint_url='http://127.0.0.1:9000/')
            ses = fs.connect()
            token = fs.get_delegated_s3pars()

I'm having this error:

Failed to generate temporary token for S3: 'coroutine' object is not subscriptable

The problem seems to be in core.py, in this section:

        async with self.session.create_client("sts") as sts:
            cred = sts.get_session_token(DurationSeconds=exp)["Credentials"]
            return {
                "key": cred["AccessKeyId"],
                "secret": cred["SecretAccessKey"],
                "token": cred["SessionToken"],
                "anon": False,
            }

I can see that get_session_token result is a coroutine (coroutine object AioBaseClient._make_api_call). Why it's trying to get the "Credentials" key from a coroutine?

Thank you for your work!

martindurant commented 1 year ago

Can you try with

        async with self.session.create_client("sts") as sts:
            cred = (await sts.get_session_token(DurationSeconds=exp))["Credentials"]
            return {
                "key": cred["AccessKeyId"],
                "secret": cred["SecretAccessKey"],
                "token": cred["SessionToken"],
                "anon": False,
            }
lamorgiac commented 1 year ago

In this way this error disappears. I’m having another problem, that is the problem that I’m facing with using directly boto3 client and get_session_token:

Unable to locate credentials

I wish I will fix it configuring right env variables, but for this issue I think your fix is right!

Cinzia La Morgia​​​​ Senior Specialist @. Piazza Trento e Trieste, 3 40137 , Bologna phone: +390516480911<tel:+390516480911> mobile: +393440248742<tel:+393440248742> www.prometeia.comhttps://www.prometeia.com/it/home @.https://www.linkedin.com/company/prometeia @.https://twitter.com/PrometeiaGroup @.https://www.instagram.com/prometeiagroup/ @.***https://www.youtube.com/channel/UCbz97h9v96HGofLJiZrScEw Please consider the environment before printing this email

Da: Martin Durant @.> Data: mercoledì, 23 agosto 2023, 16:30 A: fsspec/s3fs @.> Cc: Cinzia La Morgia @.>, Author @.> Oggetto: Re: [fsspec/s3fs] "coroutine object is not subscriptable" error calling get_delegated_s3pars (Issue #774)

Can you try with

    async with self.session.create_client("sts") as sts:

        cred = (await sts.get_session_token(DurationSeconds=exp))["Credentials"]

        return {

            "key": cred["AccessKeyId"],

            "secret": cred["SecretAccessKey"],

            "token": cred["SessionToken"],

            "anon": False,

        }

— Reply to this email directly, view it on GitHubhttps://github.com/fsspec/s3fs/issues/774#issuecomment-1690071904, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AWOD2LDBI7TP5SHM2HONKO3XWYHWZANCNFSM6AAAAAA33RZKLQ. You are receiving this because you authored the thread.Message ID: @.***>

martindurant commented 1 year ago

You are most welcome to propose that change to the code as a PR. I daresay that get_delegated_s3pars is not used much, and since it is not implemented by moto, it is not tested in CI. I am glad you have it working - but getting your credentials right is another matter.