BUG - Multi Level data in database not matching the ECMWF forecast for CNN cities

[mwalker-scottlogic]

• Exploratory testing was undertaken to investigate the accuracy of the forecast data against the forecast for given cities released by ECMWF (full details of testing methodology are available here
• PM10 & PM2.5 forecast and database values are very close, usually out by 0.00 – 0.01%
• O3 & NO2 are consistently over 10% out

Question posted to mentors about this:

• https://teams.microsoft.com/l/message/19:ffe9cf768dd54e0cacfe98b309daceaa@thread.tacv2/1716199317688?tenantId=21b711c6-aab7-4d36-9ffb-ac0357bc20ba&groupId=65943b88-8529-409f-b573-1d9452418941&parentMessageId=1716199317688&teamName=Code%20for%20Earth%202024&channelName=Channel16&createdTime=1716199317688&ngc=true

Response from Miha:

• "...I think I know what is the reason for the differences you are seeing. Your multi-level requests are correct, the issue is that the physical quantities archived at single- and multi-level are not the same. Air quality observations are usually reported as mass concentration in µg/m3. The CAMS surface single level forecasts (PM2.5 and PM10) are also archived as mass concentration but in kg/m3. Due to the internal model formulation, the multi-level fields (NO2, O3 and SO2) are archived as mass mixing ratios (MMR). To convert MMR to mass concentration you need to multiply the values with air density. See this page for details: https://forum.ecmwf.int/t/convert-mass-mixing-ratio-mmr-to-mass-concentration-or-to-volume-mixing-ratio-vmr/1253"

[mwalker-scottlogic]

Ticket updated in response to feedback from mentors

[sebsteinig]

I followed https://forum.ecmwf.int/t/convert-mass-mixing-ratio-mmr-to-mass-concentration-or-to-volume-mixing-ratio-vmr/1253 to convert those pollutants with units of mass mixing ratios to mass concentrations. For this I additionally fetch surface pressure and temperature to calculate density and multiply the mass mixing ratios by it.

Results are now nearly identical (but not exactly) to the ECMWF reported values for O3 and NO2. But deviations are only visible in the second decimal, which is more than enough precision (I guess?!). PM10 and PM2.5 are identical to the ECMWF values, so we seem to be doing the correct bilinear interpolation.

Associated PR is here: https://github.com/ECMWFCode4Earth/vAirify/pull/40

[mwalker-scottlogic]

Re-tested, there is some deviation over 0.5% in certain cities such as Los Angeles