search

nhsx / covid-chest-imaging-database

NHSX and the British Society of Thoracic Imaging (BSTI) have formed a joint partnership in order to create a national database of chest X-ray and CT images. This is to enable the validation and development of automated analysis technologies, and to promote research projects in response to the COVID-19 pandemic.
https://nhsx.github.io/covid-chest-imaging-database/
MIT License
54 stars 11 forks source link

Log all relevant env vars when running pipeline #160

Closed imrehg closed 4 years ago

imrehg commented 4 years ago

Describe the feature request

The behaviour of the pipeline is modified by the environment variables. Logging them helps understand what was happening when only seeing logs.

Affected product area (please put an X in all the brackets that apply)

[ ] Documentation [ ] Data access [ ] Infrastructure [x] Loader pipeline [ ] Other (please describe)

Additional context

While there's a pipeline run going, the env vars are visible in the AWS dashboard, but that doesn't help with post-run debugging, as task are cleared out in the interface very quickly.

imrehg commented 4 years ago

This is actually possible since #138 by setting the DEBUG env var to any value. Thus it would require a re-run if those variables are needed.

https://github.com/nhsx/covid-chest-imaging-database/blob/467d608f425d9a6f8caf2ea3a8db643cd6903518/infrastructure/container/fetch_and_run.sh#L47-L49