Prepare CLI routines for deployment via ECS - Githubissues

enram / vptstools

Python library to transfer and convert vertical profile time series data

https://enram.github.io/vptstools/

MIT License

3 stars 1 forks source link

Prepare CLI routines for deployment via ECS #45

Closed stijnvanhoey closed 1 year ago

stijnvanhoey commented 1 year ago

[x] use environmental variables instead of config files.
- FTP_HOST=xxxx
- FTP_PORT=xxxx
- FTP_USERNAME=xxxx
- FTP_PWD=xxxx
- FTP_DATADIR=data
- DESTINATION_BUCKET=aloft
- INVENTORY_BUCKET=xxxx
- SNS_TOPIC=xxxx (+ optionally export AWS_PROFILE and AWS_REGION)
[x] push info on failed runs to SNS topic when the CLI fails.

@TheJenne18, can you have a check? This PR covers the discussion in https://github.com/enram/vptstools/issues/41 (using environmental vars instead of the config or hard-coded constants) and https://github.com/enram/vptstools/issues/30 (pushing failed routines to SNS topic).

I implemented the latter as a wrapper around the CLI that catches all exceptions and pushes to SNS topic on failure. I did not yet added the mail when no files in FTP-directory. I'll add that if we know this approach works.

[x] Ignore the SNS reporting feature when running the unit tests -> pushing to mocked endpoint.

coveralls commented 1 year ago

Pull Request Test Coverage Report for Build 5382841376

43 of 58 (74.14%) changed or added relevant lines in 3 files are covered.
13 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-6.1%) to 87.5%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/vptstools/bin/click_exception.py	28	43	65.12%
<!--	Total:	43	58	74.14%	-->

Files with Coverage Reduction	New Missed Lines	%
src/vptstools/bin/vph5_to_vpts.py	13	78.31%
<!--	Total:	13		-->

Totals
Change from base Build 5050624846:	-6.1%
Covered Lines:	455
Relevant Lines:	484

💛 - Coveralls

stijnvanhoey commented 1 year ago

@peterdesmet I tested and checked the release to pypi in the test.pypi https://test.pypi.org/project/vptstools/0.1.0a14/ successfully. (I just called these releases alpha releases for testing purposes).

Merging this, so we can also integrate the deployment to AWS, see https://test.pypi.org/project/vptstools/0.1.0a14/

Note the documentation action still need to be tested and the required tokens added. The ci action just added as such