search

pwr22 / covbot

Matrix bot tracking SARS-COV-2 statistics.
https://matrix.to/#/!awMILboqesCENShaME:shortestpath.dev?via=shortestpath.dev&via=matrix.org&via=thomcat.rocks
MIT License
11 stars 1 forks source link

Give more precision on !risk #38

Open pwr22 opened 4 years ago

pwr22 commented 4 years ago

People are very confused by the 100% thing

Biep commented 4 years ago

More precision is only needed at the extremes, near 0% and 100%.

bertiebaggio commented 4 years ago

Firstly, it may be worth seeing if there are updated risk models now.

Secondly, simply adding more precision to the output won't help:

I estimate a 16 year old patient sick with COVID-19 has a 100.000000000000000% chance of survival, a 2.7% likelihood of needing to go to hospital, a 0.0% risk of needing intensive care there and a 0.000000000000000% chance of death.

(A precision of {.15%})

It is a limitation of the model used, which produces a value of -0.000966549732359168 for a 16 year old (W|A). Obviously that doesn't make sense so it is stated as 'zero risk', which of course also isn't accurate.

Perhaps we could change the wording:

The risk model estimates a 16 year old patient sick with COVID-19 has a 100% chance of survival, a 2.7% likelihood of needing to go to hospital, a 0% risk of needing intensive care there and a 0% chance of death. These values should be used with caution due to the limits of the model used.

with the bold part only printed when there is a 0/100% figure.

Alternatively:

For a 16 year old patient sick with COVID-19 the risk model estimates a 2.7% likelihood of needing to go to hospital, and is unable to give a value for the risk of death or ICU admission.

The former is more consistent with non-zero/100 outputs, but I feel the latter might be better.

pwr22 commented 4 years ago

Thanks for taking a look at this @bertiebaggio :smile:. I like your second suggestion but maybe we could take it a step further and only report "useful" figures. So it would be something like

The risk model suggests a 16 year old patient sick with COVID-19 has around a 2.7% likelihood of needing to go to hospital.

This of course has the problem that some people will consider being told they have a 100% chance of survival to be useful.

Another option is converting figures into fuzzy human terms

The risk model suggests a 16 year old patient sick with COVID-19 has an extremely good chance of surviving, a low likelihood of needing hospital treatment and an extremely low risk of needing intensive care there.

I've been mulling over this for a while but then we need to decide what the boundaries are :thinking:.

Biep commented 4 years ago

Another way could be only to report the small figures. So not mentioning the "100% survival", but mentioning the "3x10^-4 chance of death".

bertiebaggio commented 4 years ago

This of course has the problem that some people will consider being told they have a 100% chance of survival to be useful.

I agree that is something people would want to know; but as far as I can tell it's not accurate- 0% death / 100% survival only exists as a limitation of the function for calculating risk. It is only reported as 0% because we take the max() of 0 or the regression value, which for 16 year olds is a small negative value.

In those cases I would suggest wording along the lines of "risk of death is incalculable". However I would have thought with much more data now there would be another risk model, though I've yet to look.

bertiebaggio commented 4 years ago

Another way could be only to report the small figures. So not mentioning the "100% survival", but mentioning the "3x10^-4 chance of death".

Where do you get the value of 3x10^-4 ? For which age?