Incorrect cost savings in alert email

[evansd]

Copying this bug report from @brianmackenna as it deserves its own issue:

---

I have come across a bug in the alert. I receive an Islington CCG alert and it tells me

Best-case possible savings We compared this CCG's spending with the best-performing 10% on all the areas we track. If it prescribed as well as those CCGs, it could save around £200,000. Here's a list of measures, sorted by potential cost savings.

Clicking on the url brings me to the Islington CCG papge with standard measures slected on the drop down, telling me there is a £13,259 saving.

a) the figure does not close to the £200k in the email b) the selected measures are not the standard measures, although do appear to be all cost saving category (perhaps the categories when the alert was made?)

When selecting the cost-saving dashboard the Islington summary is £57k and the PPU total is £86k so still unrelated to the £200k specified in the email. The email uses "best 10%" and the dashbnoard uses median so this might explain discrepancy in the figures.

[evansd]

Don't have time to investigate this properly right now, but the cost difference is too big to be explained by median vs best decile.

This code looks a bit suspicious: https://github.com/ebmdatalab/openprescribing/blob/77d31f088418a04487c4f929ce8307baec2a9365/openprescribing/frontend/views/bookmark_utils.py#L410-L414

I suspect there might be some double counting going on, although I can't immediately think what the mechanism is. We should probably try to use the MeasureValue manager to do the filtering: https://github.com/ebmdatalab/openprescribing/blob/77d31f088418a04487c4f929ce8307baec2a9365/openprescribing/frontend/managers.py#L26-L51

[evansd]

Further thoughts. We say in the alert email:

We compared this CCG's spending with the best-performing 10% on all the areas we track. If it prescribed as well as those CCGs, it could save around £X. Here's a list of measures, sorted by potential cost savings.

But the page we link to:

• isn't sorted by potential cost savings;
• doesn't show all the measures we use to calculate the saving as it only shows "core" measures and not e.g. Low Priority ones;
• gives as a headline saving the median figure rather than best decile;
• and calculates the cost savings in a different way from the email (see also #2083).

All in all it's a pretty confusing experience.

[evansd]

Aside from the UX, the code which aggregates the savings for the email doesn't do any filtering of the measures it includes. I suspect therefore that it might be double counting the low priority savings as it will include the omnibus measures as well as the individual components.