Test setup to verify ECMWF produced forecast against our ETL output in database

[mwalker-scottlogic]

Description

This test is not fully automated yet - the process of running the main application and actually getting the ECMWF files is currently manual and requires another ticket to produce. However this can be run locally to check the divergence of cities for a specified forecast valid time from the ECMWF forecast. This is a method to indicate if we are calibrated with the current output. The tolerated divergence is set arbitrarily to 3% as present.

To run this test, you will need to download from the below links the list of CNN countries, and the desired forecast, and save the files within air-quality-backend. You then specify their path within air-quality-backend/system_tests/forecast_calibration_tests.py.

You then need to specify a matching base date for the forecast and a desired valid time to check against.

• List of stations (https://myftp.ecmwf.int/view/public/cams/products/cams_global_forecast/surface_concentrations/CAMS_locations_V1.csv) •
• Latest air quality forecasts for the list of the CNN cities in CSV format (https://myftp.ecmwf.int/files/public/cams/products/cams_global_forecast/surface_concentrations/)

air-quality-backend/system_tests/cities_data.py

• created two datasets
• all cities & a subset that have longitudes closest to -180, 0 and 180

air-quality-backend/system_tests/forecast_calibration_tests.py

• main test file
• tests are parameterised to run against desired dataset
• tests assert if database data for )3, NO2, PM10 and PM2_5 are within the desired tolerance of the ECMWF forecast

air-quality-backend/system_tests/utils/helper_methods.py

• amended functionality of the get_database_data() function and added new helper methods for the tests

Verification

• Output has been checked manually, an example below shows the test identifying that the only city above 2% divergence at a given base time and time slot for Ozone was Bogota

[github-actions[bot]]

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines	Covered	Coverage	Threshold	Status
361	249	69%	65%	🟢

New Files

No new covered files...

Modified Files

No covered modified files...

updated for commit: 159876c by action🐍

[mwalker-scottlogic]

Added changes, retested at a 1% threshold and compared to doing the check manually and the results matched