Potential issue between measures 1, 2 and 9

rcpch / rcpch-audit-engine

Epilepsy12 Audit Platform

https://e12.rcpch.ac.uk/

GNU Affero General Public License v3.0

5 stars 5 forks source link

Potential issue between measures 1, 2 and 9 #934

Closed mbarton closed 3 months ago

mbarton commented 3 months ago

Reported by @nikyraja

We need to spot check the denominators for measures 1, 2, 9a and 9b.

They should be the entire cohort but in the latest KPI report we have different values. 1 and 2 are the same, 9a and 9b are the same but they should all be the same

eatyourpeas commented 3 months ago

So just looking at this and it looks like some records the value in the aggregation table is out of date. I have found that if you find individual cases where the measure for 9A and 9B denominator does not match the total cases, the actual KPIs in the KPI table are all correct, and if you rerun them, the subsequent aggregation is correct.

eatyourpeas commented 3 months ago

As an update - I have been through all the discrepant trusts and for each one have run the calculate_kpis function against each completed case and this seems to have updated all the aggregation tables with the correct data. I think we should keep an eye on this as time goes by.

mbarton commented 3 months ago

Thanks @eatyourpeas. I think there's two take aways from this:

Scoring each case when calculating the reports rather than on update would avoid this issue
- Although this may not be possible for performance reasons
We need to get our logs into storage that survives across container restarts
- In theory we could go back through the logs trying to work out why calculate_kpis didn't happen#
- Although in practice it would be hard to find the issue at the moment we can't after a deploy
- This should be covered when we move to the new infrastructure