mweiss17
commented
4 years ago The pr for these changes is here: https://github.com/mila-iqia/covi-simulator/pull/60/
This issue makes it sounds like "carefulness is a confounder", but it is not clear to me that this is the case. Nor is it clearly the case that some of these things should be necessarily de-correlated. Sure, we don't have evidence that likliness to get tested correlates with accuracy of symptom inputs, but it might be. And I could argue that representing this as a "carefulness" attribute of the human is a reasonable way to represent this correlation.
Anyway, we're going to try it your way in that PR and see how it goes. A major advantage of de-correlation is that the simulator will be more simplistic.
TL;DR
Now, I understand the motivation behind it; but as of now,
carefulnessis a ridiculously powerful parameter and touches an insane number of things. The reason this is unhealthy is because a very large number of things in the simulator will end up being (unintentionally) correlated because they're confounded by this one variable.One example: the less likely a human is to tell the app its age and report symptoms, the more likely it'll have trouble breathing at COVID onset.
The right solution (IMO) is not to get rid of carefulness, but to break it up to a bunch of smaller parameters that can be sampled independently of each other, tuned and made sense of individually. By that, I don't mean that these smaller parameters cannot correlate with each other -- just that they should correlate only when we intend them to.
Background Stats
With P_CAREFUL_PERSON set to 0.6 in the config and a sample size of 10000:
carefulnessis 0.4 (10th percentile), 0.67 (50th percentile = median) and 0.86 (90th percentile).List of things carefulness confounds
But before that: unless otherwise stated, all numbers you'll see below are generated assuming a 40 year old with
P_CAREFUL_PERSONset to 0.6,initial_viral_loadset to 0.5 (average) and no preexisting conditions. All percentiles are assumed to be w.r.t. the carefulness samples. Also: 🤔 means "uh oh", 😱 means this, 🤯 means this.🤔 The probability that fatigue shows up in covid symptoms: at onset,
p_lethargywill generally be between 0.3 (10 %tile), 0.16 (50 %tile) and 0.07 (90 %tile). At plateau, it can be between 0.65, 0.5, 0.42.😱 The probability that trouble breathing shows up in covid symptoms: at onset, it will generally be between 0.15, 0.08 and 0.035. This is a pretty big deal, because it's a strong signal for deciding whether the human has covid (especially at onset) -- and the extent to which it's controlled by
carefulnessis scary. And this is when we control for age.🤔 How many allergy symptoms a human would show. Now, this will generally be between 3, and 2 for the model human above, but it can be sensitive to outliers in the way it's computed.
🤔 How long colds last. If carefulness is larger than 0.5, colds will last a day longer. This means you get more symptoms for longer.
🤔 How long flus last. They can last 1 to 2 days longer depending on carefulness. Again, this would indirectly interfere with the symptoms.
😱 How often symptoms are not reported (symptom dropout). Not only does this correlate age with symptom dropout (which is fine), but how many (and what) symptoms you have with symptom dropout.
😱 If the human gets tested. To reiterate -- whether the human decides to get tested correlates with all the things in this list. I think we know how many things this will in turn affect.
🤔 Whether the human wears a mask. But it doesn't look like this variable does much right now.
🤯 Whether the human logs info. This is probably the worst of the bunch, because it affects who logs their age and sex in the app, which in turn is used by the ML to infer something (latent) like carefulness. So we're essentially inducing a pretty bad sampling bias.