55 as a tester i have an automated integration test suite for the cams etl pipeline so that system quality does not regress

[mnyamunda-scottlogic]

The goal of this is as a manual regression file which can later be tweaked and added into a pipeline.

These tests run the main run_forecast_etl script but with the same grib each time. Specifically the GRIBs created on 2024-06-04 with abase_time of 00:00.

This ensures that we are uploading the exact same documents into the database every time. Therefore allowing use to assert exact keys and values.

We can confidently assert these exact values as the calculation has been done manually to see if they match the database. Bilinear interpolation etc.

Database is wiped and 'run_forecast_etl' is initiated with overridden "FORECAST_BASE_TIME"

Tests include

• Querying unique documents which should only bring back a single document.
  • Decision to query locations that are close to boundaries of -180 , 0, and 180 Vancouver, London and Wellington respectively.
  • Assert exact values that ideally shouldn't change as we are using the same grib file.
• Assert that the aqi values can only be between 1 and 6 inclusive for both overall_aqi_levelsand also individual_pollutant_aqi
• Assert that the 'overall_aqi_level' must be the result of the highest value displayed by 'individual_aqi_levels' for example if values show 1 3 3 3 1 then the 'overall_aqi_level' must display 3
• Assert that only location_type "city" applies to all documents
• Assert that query gets back exactly 6273 documents for our 'base_time' of 00:00
• Assert that created time roughly matches the time at which the user ran the script from start to finish

[github-actions[bot]]

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines	Covered	Coverage	Threshold	Status
764	715	94%	80%	🟢

New Files

No new covered files...

Modified Files

No covered modified files...

updated for commit: 09f78a6 by action🐍