search

SneaksAndData / adapta

Logging, data connectors, monitoring, secret handling and general lifehacks to make data people lives easier.
Apache License 2.0
8 stars 1 forks source link

[BUG] In version 3.2 S3StorageClient expects PROTEUS__AWS_SECRET_ACCESS_KEY #470

Open henrikfoss opened 3 weeks ago

henrikfoss commented 3 weeks ago

Previously I could run:

storage_client = S3StorageClient.create(auth=auth)

with 

s3_client = AwsClient(
    aws_credentials=ExplicitAwsCredentials(
        access_key=secrets["sa_secret_access_key"],
        access_key_id=secrets["sa_access_key"],
        region=region,
        endpoint=endpoint,
    )
)

But after upgrading to 3.2, it is giving me this error:

File "/home/hfj/.cache/pypoetry/virtualenvs/omni-channel-crystal-solver-BUzSMOZ0-py3.11/lib/python3.11/site-packages/IPython/core/interactiveshell.py", line 3553, in run_code exec(code_obj, self.user_global_ns, self.user_ns) File "", line 1, in <cell line: 0> runfile('/home/hfj/PycharmProjects/omni-channel-crystal-solver/examples/main_local.py', wdir='/home/hfj/PycharmProjects/omni-channel-crystal-solver/examples') File "/opt/pycharm-2023.1.2/plugins/python/helpers/pydev/_pydev_bundle/pydev_umd.py", line 198, in runfile pydev_imports.execfile(filename, global_vars, local_vars) # execute the script ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/pycharm-2023.1.2/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile exec(compile(contents+"\n", file, 'exec'), glob, loc) File "/home/hfj/PycharmProjects/omni-channel-crystal-solver/examples/main_local.py", line 132, in args = mock_arguments(file) ^^^^^^^^^^^^^^^^^^^^ File "/home/hfj/PycharmProjects/omni-channel-crystal-solver/examples/main_local.py", line 102, in mock_arguments test_uri = _get_test_data_sas_uri(payload_data, auth=s3_client) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/hfj/PycharmProjects/omni-channel-crystal-solver/examples/main_local.py", line 31, in _get_test_data_sas_uri storage_client = S3StorageClient.create(auth=auth) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/hfj/.cache/pypoetry/virtualenvs/omni-channel-crystal-solver-BUzSMOZ0-py3.11/lib/python3.11/site-packages/adapta/storage/blob/s3_storage_client.py", line 63, in create auth.initialize_session(session_callable) File "/home/hfj/.cache/pypoetry/virtualenvs/omni-channel-crystal-solver-BUzSMOZ0-py3.11/lib/python3.11/site-packages/adapta/security/clients/aws/_aws_client.py", line 106, in initialize_session self._session = session_callable() ^^^^^^^^^^^^^^^^^^ File "/home/hfj/.cache/pypoetry/virtualenvs/omni-channel-crystal-solver-BUzSMOZ0-py3.11/lib/python3.11/site-packages/adapta/security/clients/aws/_aws_client.py", line 115, in _default_aws_session self._credentials = EnvironmentAwsCredentials() ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/hfj/.cache/pypoetry/virtualenvs/omni-channel-crystal-solver-BUzSMOZ0-py3.11/lib/python3.11/site-packages/adapta/security/clients/aws/_aws_credentials.py", line 64, in init raise ValueError("PROTEUS__AWS_SECRET_ACCESS_KEY must be set") ValueError: PROTEUS__AWS_SECRET_ACCESS_KEY must be set

jrbentzon commented 3 weeks ago

We updated the constructor argument from aws_credentials to credentials, can you try 

s3_client = AwsClient(
    credentials=ExplicitAwsCredentials(
        access_key=secrets["sa_secret_access_key"],
        access_key_id=secrets["sa_access_key"],
        region=region,
        endpoint=endpoint,
    )
)