The-Strategy-Unit / nhp_project_information

Project summary and technical documentation for the New Hospitals Programme demand and capacity tool
https://connect.strategyunitwm.nhs.uk/nhp/project_information/
Other
2 stars 0 forks source link

Guidance on prediction intervals for schemes #127

Open StatsRhian opened 1 month ago

StatsRhian commented 1 month ago

Apparently some schemes are adding together P10/ P90 values to get prediction intervals across multiple rows in the spreadsheet. This is not correct, you can’t add together distributional values.

I’ve spoken to Jake/ Karen/ Peter today and the suggestion is:

Update the guidance https://connect.strategyunitwm.nhs.uk/nhp/project_information/user_guide/glossary.html#prediction-intervals to be a bit stronger on that adding these values together is wrong, rather than just different from what’s observed in the S curve

Provide some R code in the documentation so they have an option to do this themselves. Simple example from Mohammed

# Given values for two distributions (P10, mean, P90)
P10_A <- 54   # example value for P10 of A
Mean_A <- 70  # example value for mean of A
P90_A <- 86   # example value for P90 of A

P10_B <- 63   # example value for P10 of B
Mean_B <- 80  # example value for mean of B
P90_B <- 95   # example value for P90 of B

# Calculate standard deviation for each distribution
z_p10 <- qnorm(0.10)
z_p90 <- qnorm(0.90)

# Standard deviation for distribution A and B
sd_A <- (P90_A - P10_A) / (z_p90 - z_p10)
sd_B <- (P90_B - P10_B) / (z_p90 - z_p10)

# Sum of means
Mean_sum <- Mean_A + Mean_B

# Standard deviation of the sum of two independent distributions
sd_sum <- sqrt(sd_A^2 + sd_B^2)

# Calculate P10 and P90 for the summed distribution
P10_sum <- Mean_sum + z_p10 * sd_sum
P90_sum <- Mean_sum + z_p90 * sd_sum

# Output results
list(
  Mean_sum = Mean_sum,
  sd_sum = sd_sum,
  P10_sum = P10_sum,
  P90_sum = P90_sum
)

# incorrect percentiales
list(
  P10_A_inccorect_sum=P10_A+P10_B,
  P90_B_inccorect_sum=P90_A+P90_B
)
StatsRhian commented 4 weeks ago

I'm going to write a short lay summary and a worked example (like we did for the interaction term) for project information and share with MRMs and Mohammed / Steven

StatsRhian commented 1 week ago

Just add the code initially.

StatsRhian commented 6 days ago

I had a go at running this code on the example and I wasn't getting the same numbers (which I think is probably right but I confused myself)

Can look again but ideally with a bit more headspace!