search

corona-warn-app / cwa-wishlist

Central repository to collect community feature requests and improvements. The CWA development ends on May 31, 2023. You still can warn other users until April 30, 2023. More information:
https://coronawarn.app/en/faq/#ramp_down
Apache License 2.0
105 stars 14 forks source link

Open Source booster notification rules #684

Closed Ein-Tim closed 2 years ago

Ein-Tim commented 2 years ago

Feature description

The booster notification rules, which are currently effective, can't be seen publicly on GitHub (or on any other place known to me).

However, for transparency reasons, they should be made open source. This helps me and everyone else to better understand who's currently getting a notification and who's not.

Problem and motivation

As the situation reg. who is allowed to get a booster & what the STIKO recommends changes quite fast, it's currently very hard to keep up with in which scenarios a notification is send.

Internal Tracking ID: EXPOSUREAPP-10984

svengabr commented 2 years ago

Internal Tracking ID: EXPOSUREAPP-10984

svengabr commented 2 years ago

Thanks, @Ein-Tim, I did create a ticket and forwarded it to the team.

If the code cant be made public for some reason, I have asked to add it to the documentation.

thomasaugsten commented 2 years ago

You can see the rules at the moment here: https://distribution.dcc-rules.de/bnrules/ https://distribution.dcc-rules.de/bnrules/31ab71cc2aa7e54e075b0b11d68b78fff33b12fbed185036d61b02649c3c63aa

Ein-Tim commented 2 years ago

For everybody interested in this, the booster notification rules are available here: https://distribution.dcc-rules.de/bnrules/

Just copy the hash of a rule & paste it after /bnrules/ and you will find the description & details of the rule.

Please note that this is not the endpoint the app uses to download the rules, the URL the app is using is https://svc90.main.px.t-online.de/version/v1/booster-notification-rules.

Still, I would really like to see the booster notification rules published here on GitHub, so that it's very easy to find them.

Thanks to @vaubaehn for pointing me to the links I shared above!

Ein-Tim commented 2 years ago

Thanks to you @thomasaugsten & @svengabr!

Ein-Tim commented 2 years ago

Meanwhile, the BNR are very out of date (https://github.com/corona-warn-app/cwa-documentation/issues/754). I don't understand why they don't get updated or, if there is no intention to update them, removed.

If the rules where open-source, the community could suggest changes through PRs.

vaubaehn commented 2 years ago

Hi @dsarkar , could you please add this issue to EXPOSUREAPP-11340? Thank you!

@thomasaugsten do you see a chance that the pipeline for the booster notifications can be extended in such a way, that the JSONLogic of the rules gets its own repo? Then the community could propose changes to the rules there or even create PRs, so that changed rules can be merged and deployed after review.

thomasaugsten commented 2 years ago

In the current approval process it is not possible at the moment to integrate external sources.

vaubaehn commented 2 years ago

@thomasaugsten

In the current approval process it is not possible at the moment to integrate external sources.

Yes, but maybe in the future that could be made possible? Same suggestion would be true for upcoming admission rules. I'm very optimistic that community can contribute and reduce delays.

vaubaehn commented 2 years ago

In a naive thinking, it would mean that the people who create/change the JSONLogic make their work files visible via a repo. If they need approval for their work, it could be sent from there. In this way, community could 'inject' their support.

MikeMcC399 commented 2 years ago

The process is currently intransparent. The rules are out of date and there is no feedback on the plans to implement newer rules:

thomasaugsten commented 2 years ago

We cannot give details about the process because we are not directly involved, please ask the publisher of the rules for details.

To copy paste the content of the stiko text is only a minor task and will not speed up the process if is done by the community.

I would be more helpful to have structured table of the 2G, 3G, 3G+, 2G+ and 2G+ + B definition for each federal state in Germany because my information this is not available at the moment and needed in many discussions

MikeMcC399 commented 2 years ago

@thomasaugsten

We cannot give details about the process because we are not directly involved, please ask the publisher of the rules for details.

Can you tell us who the publisher of the rules is? How can we contact the publisher?

vaubaehn commented 2 years ago

Thanks for your response, @thomasaugsten . Just one last short question for now:

We cannot give details about the process because we are not directly involved, please ask the publisher of the rules for details.

Are you actually talking about the publisher (RKI) or the company that hosts/deploys the booster notification rules to adress such a request?

To copy paste the content of the stiko text is only a minor task and will not speed up the process if is done by the community.

I was thinking about taking care about the whole rule set (JSONlogic), not only altering the description in accordance with the STIKO recommendation. But you're right, for the booster notifications this is "überschaubar", but as we can see in practice, they're not maintained in the necessary speed. This is why I think, the community could help out here.

I would be more helpful to have structured table of the 2G, 3G, 3G+, 2G+ and 2G+ + B definition for each federal state in Germany because my information this is not available at the moment and needed in many discussions

This is actually a problem. I don't know if there is an official institution at all to track this. In worst case, maybe we can try to gather these information as a community effort - @Ein-Tim ?

Edit: Tim, I wasn't mentioning you to do the work, but to get your opinion. But if we try something like this, it could be helpful to ping your contacts on Twitter to engage in this work.

Ein-Tim commented 2 years ago

@vaubaehn

In worst case, maybe we can try to gather these information as a community effort

Phew, that's a hard thing especially as this changes nearly weekly. I would not be confident with even giving the rules for my own federal states at the moment. 😅

vaubaehn commented 2 years ago

@vaubaehn

In worst case, maybe we can try to gather these information as a community effort

Phew, that's a hard thing especially as this changes nearly weekly. I would not be confident with even giving the rules for my own federal states at the moment. 😅

@Ein-Tim Right, and me neither... But if we get some people from every federal state, that could be achieved, maybe. But I see the difficulties... But my gut feeling is, we won't get anything like this from RKI/BMG...

vaubaehn commented 2 years ago

Maybe we can put something like this on GH: https://docs.google.com/spreadsheets/d/1W2Y0nUhRAwLr1LFEMIQogsIr3-08Is_FVPuAHKLiZY8/edit?usp=sharing

I will look whether in "Darf ich ~rein~ das?" information about admission rules is provided.

thomasaugsten commented 2 years ago

@vaubaehn Correct this are right contact for this RKI you can reach out impfnachweis@rki.de I think if there a public repo available a lot people will help here to build up a long required list

Ein-Tim commented 2 years ago

@vaubaehn Please write me a DM via Slack, then we can talk about how we could start this.

vaubaehn commented 2 years ago

@thomasaugsten Thank you! @Ein-Tim I will do!

Bergreiter commented 2 years ago

I like the idea I started a basic idea maybe this a good start https://github.com/Bergreiter/ZugangsregelnDeutschland

MikeMcC399 commented 2 years ago

@Ein-Tim

Regarding the original request:

The booster notification rules, which are currently effective, can't be seen publicly on GitHub (or on any other place known to me).

@dsarkar shared a script in https://github.com/corona-warn-app/cwa-documentation/issues/754#issuecomment-999644069 to extract the rules.

vaubaehn commented 2 years ago

Hi @Bergreiter and all,

thanks for your proposal regarding your repo ZugangsregelnDeutschland. Having a space on GitHub has advantages (public access, associated with the project) and disadvantages (concept of collaboration known to less users), and could be discussed further, though personally I tend to a GitHub-solution. In the end I think it would be good to have an official repo that is directly embedded into the CWA project, but accessable from the public. A table that we have started could result in the necessary admission rules finally. While trying a draft on a basic concept what information is needed to build up a table, for me it seems many information is nested:

So we first need to discuss which information we need and how they could be ordered best to be able to fill out table cells in the end.

After I wrote above, I'll have a look into 'darf ich das', I had a look... And indeed some information are available, but they're not ordered in a comfortable way. I then saw, that the company maintaining the app is taking their data from Tourismus-wegweiser.de which is maintained by Project M GmbH Hamburg - and they're contracted by the Ministry of Economy and Energy. So maybe it's worth to officially ask there whether they already have structured data according admission rules and certificate rules? And connect to RKI to get a repo for admission rules as well as booster notification rules embedded into the CWA project?

Finally I suggest that we start a new issue for all further details about admission rules, as the issue here deals with booster notification rules.

How are your thoughts?

vaubaehn commented 2 years ago

@thomasaugsten and @mlenkeit The Kompetenzzentrum Tourismus des Bundes provides a public JSON-Api: https://tourismus-wegweiser.de/json For sure, this data doesn't help. But it looks like they have tools to create this JSON... And if they have already structured data about admission rules, that could become a way to access these data also for CWA/CovPass(Check). At least it's worth to engage in a joint effort/collaboration. Maybe you can ask RKI/BMG whether this is possible?

dsarkar commented 2 years ago

German Acceptance Rules (Business Rules)

Booster Notification Rules

Script - German Acceptance Rules (Business Rules)

for i in $( wget -qO- https://distribution.dcc-rules.de/rules/DE/   | jq -r '.[].hash' ) 
do 
  json=$(   wget -qO- https://distribution.dcc-rules.de/rules/DE/$i) 
  Identifier=$(     echo $json | jq -r '.Identifier')
  Type=$(           echo $json | jq -r '.Type')
  Country=$(        echo $json | jq -r '.Country')
  Version=$(        echo $json | jq -r '.Version')
  SchemaVersion=$(  echo $json | jq -r '.SchemaVersion')
  Engine=$(         echo $json | jq -r '.Engine')
  EngineVersion=$(  echo $json | jq -r '.EngineVersion')
  CertificateType=$(echo $json | jq -r '.CertificateType')
  ValidFrom=$(      echo $json | jq -r '.ValidFrom')
  ValidTo=$(        echo $json | jq -r '.ValidTo')
  desc_de=$(        echo $json | jq -r '.Description[1].desc')
  desc_en=$(        echo $json | jq -r '.Description[0].desc')

  echo "- **"$Identifier"**" $Type $Country $Version $SchemaVersion $Engine $EngineVersion $CertificateType $ValidFrom $ValidTo
  echo "  - **DE** $desc_de"
  echo "  - **EN** $desc_en"
  echo "  - [$i](https://distribution.dcc-rules.de/rules/DE/$i)"
done

Script to get Booster Notification Rules

for i in $( wget -qO- https://distribution.dcc-rules.de/bnrules    | jq -r '.[].hash' ) 
do 
  json=$(   wget -qO- https://distribution.dcc-rules.de/bnrules/$i) 
  Identifier=$(     echo $json | jq -r '.Identifier')
  Type=$(           echo $json | jq -r '.Type')
  Country=$(        echo $json | jq -r '.Country')
  Version=$(        echo $json | jq -r '.Version')
  SchemaVersion=$(  echo $json | jq -r '.SchemaVersion')
  Engine=$(         echo $json | jq -r '.Engine')
  EngineVersion=$(  echo $json | jq -r '.EngineVersion')
  CertificateType=$(echo $json | jq -r '.CertificateType')
  ValidFrom=$(      echo $json | jq -r '.ValidFrom')
  ValidTo=$(        echo $json | jq -r '.ValidTo')
  desc_de=$(        echo $json | jq -r '.Description[1].desc' | sed /^$/d)
  desc_en=$(        echo $json | jq -r '.Description[0].desc' | sed /^$/d)

  echo "- **"$Identifier"**" $Type $Country $Version $SchemaVersion $Engine $EngineVersion $CertificateType $ValidFrom $ValidTo
  echo "  - **DE** $desc_de"
  echo "  - **EN** $desc_en"
  echo "  - [$i](https://distribution.dcc-rules.de/bnrules/$i)"
done
vaubaehn commented 2 years ago

Great work, @dsarkar . I had a look, who probably have their hands on coding the business rules (via dgc repo - business rules test data) as they're similarly outdated (not yet adopted for rules taking effect Feb 1st) and as I assume the same persons code the booster notification rules. The last change for a business rule was VR-DE-0003 in the test data repo... Maybe someone (else) can ping that person when it comes to questions regarding updating/correcting business/booster rules?

vaubaehn commented 2 years ago

Hi @thomasaugsten , Tim and me are going to prepare an eMail to RKI like you suggested in https://github.com/corona-warn-app/cwa-wishlist/issues/684#issuecomment-1020362571. Would you be fine to be in cc, so that a response can directly reach you?

thomasaugsten commented 2 years ago

I'm not involved in the process no need to cc me here.

vaubaehn commented 2 years ago

Hi @thomasaugsten , one question regarding the booster notification rules: Will the "old" booster-notification-rules-system be deprecated soon, and be merged with the "new CCL", or will BNRs and CCLs exist in parallel?

thomasaugsten commented 2 years ago

They will exist in parallel the client will download BNR and the CCL will apply the BNR on the existing certificates

vaubaehn commented 2 years ago

@thomasaugsten : Thank you - that information helps to refine the mail to RKI.

MikeMcC399 commented 2 years ago

@vaubaehn Could you explain the term "CCL" for those (like me) not familiar with it? Is it a set of rules or is it a rule engine maybe?

mlenkeit commented 2 years ago

@MikeMcC399

Common Covid Logic (CCL) describes a layer of common logic - especially regarding the interpretation of Digital COVID Certificates (DCCs) - that can be shared across different operating systems and implementations.

The logic in CCL is described in JSON format. An engine processes the logic on the client against a defined input structure (e.g. the set of DCCs) and can expect a defined output structure. The client can update the JSON regularly from a server. This facilitates updates to CCL independent of client releases, e.g. in order to respond to legal or regulatory pandemic-related changes on short notice.

That's what the repo will say, will be available soon...

MikeMcC399 commented 2 years ago

@mlenkeit Many thanks for educating us about the meaning and plans for CCL! Sounds excellent!

Ein-Tim commented 2 years ago

@MikeMcC399

FYI: https://github.com/corona-warn-app/cwa-app-ccl

(cc @mlenkeit & @vaubaehn)

vaubaehn commented 2 years ago

@mlenkeit @thomasaugsten Don't have time, but couldn't wait to have a look - and it looks 🎉 and ❤️ ! Short question: Rule name/type "IR" -> is this for admission rules? ("Immunization Rule"?)

Ein-Tim commented 2 years ago

@thomasaugsten

I would be more helpful to have structured table of the 2G, 3G, 3G+, 2G+ and 2G+ + B definition for each federal state in Germany because my information this is not available at the moment and needed in many discussions

Maybe this gives you a starting point: https://www.dehoga-corona.de/fileadmin/user_upload/UEbersicht_Bundeslaender_Ausnahme_von_Testpflicht__2G-Plus__insbesondere_fuer_Geboosterte_2022_01_26_12-00.pdf

mlenkeit commented 2 years ago

Short question: Rule name/type "IR" -> is this for admission rules? ("Immunization Rule"?)

@vaubaehn IR is the abbreviation for Invalidation Rule, a mechanism that allows to invalidate DCCs, e.g. "vaccination certificates issued by Issuer A between t0 and t1 are not valid".

vaubaehn commented 2 years ago

Short question: Rule name/type "IR" -> is this for admission rules? ("Immunization Rule"?)

@vaubaehn IR is the abbreviation for Invalidation Rule, a mechanism that allows to invalidate DCCs, e.g. "vaccination certificates issued by Issuer A between t0 and t1 are not valid".

Good morning, @mlenkeit , thanks for your response. It's a bit confusing, as we had the rule type "Invalidation Rule" before, although it wasn't used, and there has not been any identifier for it. I'm referring to https://github.com/corona-warn-app/cwa-app-ccl/blob/2ae52b6d2b82131eda00984f8298e1062affddcb/resources/json-schema/dcc-validation-rule.json#L22-L36 where we see 'IR' as a rule identifier, and like 'Acceptance' (that never had any identifier) the 'Invalidation' is a rule type... Is the "new" invalidation rule for upcoming feature of blocking fraudulent DCC hashes then? And how are the 'admission rules' (rules to calculate an admission state relying on DccWalletInfo) entering the CCL? Are they coming as/together with rule type BNR? Sorry, I hope I'm not stealing too much of your time.

Ein-Tim commented 2 years ago

This is quite a good start: https://github.com/corona-warn-app/dcc-rule-translation

Let's see how this evolves, if all notifications can be seen there I will close this issue.

Ein-Tim commented 2 years ago

This has been de-facto implemented with https://github.com/corona-warn-app/dcc-rule-translation. Therefore closing this issue.