Migrate covid-utils CLI tool to use Embassy S3 and not AWS S3

[dipayan1985]

The covid-util CLI tool uses IAM users to access the underlying S3 infrastructure. I have requested for IAM users COVIDAdmin and COVIDUser.

Creating a IAM user is very easy and can be done in minutes in AWS. Not so sure about Embassy. I had a chat with David and he says that as we are the first requestor of S3 services, we are also the first requestor of this IAM configuration. They will try to figure it out. I hope not but it might take some time as well.

/cc @mcourtot

[dipayan1985]

We have received the Embassy users now. I will do the changes in the covid-util CLI tool.

[dipayan1985]

@z-w123 , @nadimm-rahman , @alisha246

The covid-util CLI tool has been forked and migrated to Embassy S3. Details below:

Deployment: For test purposes I have deployed it in test pypi at https://test.pypi.org/project/covid-util-embassy/

Note: Please don’t use the command - pip install covid-util-embassy, the project is still in test pypi. The correct command will be

python3 -m pip install --index-url https://test.pypi.org/simple/ --no-deps covid-util-embassy or pip install -i https://test.pypi.org/simple/ covid-util-embassy

Rest of the commands and their options remain the same as covid-util which you are currently using, except the command line is named as covid-util-embassy and not covid-util.

Readme is here - https://github.com/EBIBioSamples/hca-util/tree/covid-util

If testing is OK I think we can deploy our uploader integrated with Embassy S3 to the COVID-19 data portal in its upcoming sprint. We will also promote the covid-util-embassy CLI to pypi from test pypi.

Please let me know what you think.

/cc @mcourtot , @Jeena-Rajan

[mcourtot]

I think this is now complete?

[dipayan1985]

@mcourtot This is complete, closing the ticket.