search

enram / vptstools

Python library to transfer and convert vertical profile time series data
https://enram.github.io/vptstools/
MIT License
3 stars 1 forks source link

Should we use production bucket and inventory? #59

Closed peterdesmet closed 1 year ago

peterdesmet commented 1 year ago

I noticed some links refer to:

  1. The aloft bucket, which will be removed. The aloftdata bucket should be used

  2. The aloft-inventory bucket (with subfolder aloft/aloft-hdf5-files-inventory/). This bucket is on UAT. I wonder if the inbo-aloft-prod-eu-west-1-inventory bucket (with subfolder /aloftdata/aloftdata-hdf5-files-inventory/) should be used instead or if that will break tests.

peterdesmet commented 1 year ago

For the 'default' environmental variable when not provided (os.environ.get("DESTINATION_BUCKET", "aloft")) I would not use the prd version but the dev/uat bucket name and sns channels

So that would then be inbo-aloft-uat-eu-west-1-default

For the s3_url_h5 I think it is ok to use the aloftdata prd bucket name

Ok

In the Examples sections of the docstrings I would sue the aloftdata prd name

Ok

For the tests/ files it should not matter as the S3-bucket is mocked for the tests. So you can choose whatever name you prefer but at the moment some tests did fail due to the name adjustments in the tests (you also adjust the dummy_manifest.json file, https://github.com/enram/vptstools/blob/main/tests/data/inventory/dummy_manifest.json#L9)

I will then use dummy-aloftdata and dummy-inventory

peterdesmet commented 1 year ago

Updated, but some tests are still failing, so I probably changed something I shouldn't have?

stijnvanhoey commented 1 year ago

so I probably changed something I shouldn't have?

Main issue was that the default env vars were used in the end2end tests of the CLI commands instead of the dummy-aloftdata and dummy-inventory. I adjusted this and made sure other occurrences of the bucket name use the dummy-aloftdata naming.