`DataAdapter` sets s3 storage options as anonymous by default crashing any data catalog with s3 credentials - Githubissues

Deltares / hydromt

HydroMT: Automated and reproducible model building and analysis

https://deltares.github.io/hydromt/

MIT License

66 stars 27 forks source link

`DataAdapter` sets s3 storage options as anonymous by default crashing any data catalog with s3 credentials #623

Open Jaapel opened 10 months ago

Jaapel commented 10 months ago

HydroMT version checks

[X] I have checked that this issue has not already been reported.
[X] I have checked that this bug exists on the latest version of HydroMT.

Reproducible Example

Use any data catalog item without storage_options and filesystem=s3 which requires credentials.

Current behaviour

errors out with a 404

Desired behaviour

https://github.com/Deltares/hydromt/blob/main/hydromt/data_adapter/data_adapter.py#L380 causes the issue Should be resolved by not letting anon=True be the default, because it is not the default for any AWS library.

Additional context

No response

Jaapel commented 10 months ago

Just discussed with @savente93 and we work with a credential chain:

try and access using anonymous credentials
then try and access AWS credentials chain
otherwise raise

As you can work around this issue by specifying anon: false in the DataCatalog, this issue has lower priority.