PersonalDataIO / CoronaRiskScoring

4 stars 2 forks source link

Are the Configuration Parameters of the Danish SmitteStop app available? #21

Open mhoehle opened 3 years ago

mhoehle commented 3 years ago

Recent news coverage have revealed some problem with the parametrisation of the Danish Smittestop App, which is based on the GAEN framework.

From reading the article information, it sounds like they had a poor choice of Exposure Configuration parameters for the App, which they had to change. However, AFAIK, the source code of the Danish app is not publically available. So my questions would be

  1. is #16 known for the Danish Smittestop app?
  2. can the parameters of, e.g.,
    ENExposureConfiguration.attenuationLevelValues,
    ENExposureConfiguration.daysSinceLastExposureLevelValues,
    ENExposureConfiguration.durationLevelValues,
    ENExposureConfiguration.transmissionRiskLevelValues and
    ExNExposureConfiguration.metadata.attenuationDurationThresholds

    be deduced in other ways?

  3. If these are already monitored, has there been a change in these parameters around 03 Oct 2020? That is parameters corresponding to for example https://github.com/corona-warn-app/cwa-server/blob/master/services/distribution/src/main/resources/master-config/exposure-config.yaml for the German app.
pdehaye commented 3 years ago

The parameters are available by running (courtesy of @sftcd over here):

curl -o dk-cfg.json -D - -L "https://app.smittestop.dk/API/v1/diagnostickeys/exposureconfiguration" -H "Authorization_Mobile: 68iXQyxZOy"

The current setup is

{"minimumRiskScore":80,"attenuationScores":[1,2,3,4,5,6,7,8],"attenuationWeight":50,"daysSinceLastExposureScores":[1,1,1,1,1,1,1,1],"daysSinceLastExposureWeight":50,"durationScores":[1,1,1,1,4,4,4,5],"durationWeight":50,"transmissionRiskScores":[1,1,8,8,8,8,1,1],"transmissionRiskWeight":50,"durationAtAttenuationThresholds":[63,73]}

and it looks to be that way since 20200929-0900 UTC.

From 20200915 to 20200929-0900 UTC the values were

{"minimumRiskScore":192,"attenuationScores":[1,1,8,8,8,8,8,8],"attenuationWeight":50,"daysSinceLastExposureScores":[1,1,1,1,1,1,1,1],"daysSinceLastExposureWeight":50,"durationScores":[1,1,1,1,8,8,8,8],"durationWeight":50,"transmissionRiskScores":[1,1,8,8,8,8,1,1],"transmissionRiskWeight":50,"durationAtAttenuationThresholds":[63,73]}

From 20200831 to 20200915 the values were:

{"minimumRiskScore":512,"attenuationScores":[1,1,8,8,8,8,8,8],"attenuationWeight":50,"daysSinceLastExposureScores":[1,1,1,1,1,1,1,1],"daysSinceLastExposureWeight":50,"durationScores":[1,1,1,1,8,8,8,8],"durationWeight":50,"transmissionRiskScores":[1,1,8,8,8,8,1,1],"transmissionRiskWeight":50,"durationAtAttenuationThresholds":[63,73]}

From 20200706 (latest) to 20200831 the values were:

{"minimumRiskScore":512,"attenuationScores":[1,1,1,8,8,8,8,8],"attenuationWeight":50,"daysSinceLastExposureScores":[1,1,1,1,1,1,1,1],"daysSinceLastExposureWeight":50,"durationScores":[1,1,1,1,8,8,8,8],"durationWeight":50,"transmissionRiskScores":[1,1,8,8,8,8,1,1],"transmissionRiskWeight":50,"durationAtAttenuationThresholds":[51,63]}
mhoehle commented 3 years ago

This is awesome! Thank you very much!

It appears that they had some problem with the application of the minimumRiskScore,but 8 8 8 >= 512 (attenuationScores x durationScores x transmissionRiskScores) , so I'm not completely sure how that became a problem. However, two things come to mind:

  1. About the transmissionRiskScores":[1,1,8,8,8,8,1,1], do you happen to know, if the SmitteStop app uses a Transmission Risk Level (TRL) as part of the Diagnosis Keys? Or if they use the same value everywhere? Otherwise, it appears strange to map values of 7 and 8 to 1...

  2. Any chance that one can also fetch the weights for the attenuation-time bucket summation attenuationBucketWeights and the risk-score-normalization-divisor (example from German CWA). This is post minimumRiskScore, but maybe this is where the score became too low...

Update after posting: I've tried to give some more context for the meaning of the parameters in this blog post about Risk Scoring in Digital Contact Tracing Apps.

pdehaye commented 3 years ago

For question 2., you had originally the thresholds {51, 63} and a weight attenuation of 50. I understand this to mean that seconds under 51 db count in full, those above 63 don't count, and those in between 100%. It is now at {63,73}, a quite substantial change.

For question 1., here is a quick sample. On 20201003-010000, in file timestamped 2020-09-30 (there is a slight delay for those to start filling in), these were the counts for TEKs with the following Transmission Risk Levels: 1: 363 2: 64 3: 70 4: 335 5: 87 6: 9 7: 8 8: 2

mhoehle commented 3 years ago

Thanks for providing the TRLs and pointing out the change in the durationAtAttenuationThresholds.

I'm slightly puzzled by the TRLs, because I don't really know what the values mean. They do not seem to have a linear order, because lvl 1-2 and lvl 7-8 is set to 1.

AFAIK, the attenuationWeight relates to the weight given to the attenuation component in the attenuation x days x TRL x duration formula. I don't think the weights are actually used in the GAEN v1. The bucket weights are likely specified somewhere else or there are default values, the German Corona Warn App uses weights of 1.0, 0.5, and 0.

To me it seems that the major problem after 20200915 was that the minimum risk score was too high. Since all four multiplied risk scores are either 1 or 8 it takes three 8's to go beyond 192. Because the days parameter is always 1, it means that all three others have to be 8 (TRL 3-6, duration >= 15 min, attenuation ≤63).

Still thinking about how this matches the explanations in this dr.dk article. Where it says (GoogleTranslate):

By default, it is your weighted, average exposure to an infected person that determines whether you are notified if one of your contacts has been infected.

They write that if you had a lot of meetings with the same person, and many of the meetings fell below the minimum risk, then these "zero meetings" average out any high risk encounters you had with the person. That statement can only relate to the attenuation-time bucket summation, where the ≤51-≤63 dB encounters previously were only counted half and the >63 dB were ignored. So maybe because of the minimumScore, too much was put to zero and, hence, did not contribue as part of the bucket summaiton... Not a smoking gun, but at least a plausible explanation.

Thanks again for your help!