lshtm-gis / WHO_PHSM_Cleaning

Cleaning PHSM provider data for WHO
https://lshtm-gis.github.io/WHO_PHSM_Cleaning/html/
MIT License
0 stars 1 forks source link

updates changes for EURO #171

Closed todowede closed 3 years ago

todowede commented 3 years ago

Updated the following codes - mainly to read the raw EURO data in excel without any preprocessing.

Include all EURO data columns header in the columns config file

config/input_check/columns/EURO.csv

Update the key_map values for EURO data to recognise the new column headers

config/key_map/EURO.csv

Update new who_code combinations for EURO

config/who_coding/EURO.csv

Include openpyxl>=3.0.6 in package requirements (required as engine for pandas.read_excel)

requirements.txt

Change of date to read previous_update for weekly processing run

src/master.py

Update to the function weekly_import for the progressive upload of EURO data

src/postprocess/EURO.py

Change of date to read previous_update and previous_not_cleansed for weekly processing run

src/prepare_merge.py

Read the raw excel file from EURO (comment out previous code that read csv file)

src/preprocess.py

Apply utils.replace_conditional for manual country name change for CDC

src/processing/CDC_ITF.py

Update code to accommodate mixed enforcement type in EURO data

src/processing/EURO.py

write test for function weekly_import that upload only who_code in euro_weekly_upload config

tests/postprocessing/test_postprocess_EURO.py