EFGS Federation Gateway support

google / exposure-notifications-server

Exposure Notification Reference Server | Covid-19 Exposure Notifications

https://www.google.com/covid19/exposurenotifications/

Apache License 2.0

2.45k stars 312 forks source link

EFGS Federation Gateway support #965

Closed marekaf closed 3 years ago

marekaf commented 4 years ago

Hello 👋 Do you plan on supporting EFGS project somehow? (open sourced with docs here, architecture spec here)

Briefly, the project is described as

The least complex and most robust way to connect the backends behind all the different
national proximity tracing apps is a Federation Gateway Service, which accepts diagnosis
keys from all countries, buffers them temporarily, and provides them for all countries to be
downloaded. Additionally, all backends can be informed immediately if new data is available,
so that transmission lags are kept minimal.

We are currently evaluating how can we extend key server's functionality to support EFGS and it would be great to know if there's any support / feature (in this repo or another) from Google already.

Thanks!

mikehelmick commented 4 years ago

Hello -

We haven't had any requests yet to support this. If it is needed in your jurisdiction, it is something we can evaluate.

Let us know if this is a blocker for you and what kind of timeline would be needed.

marekaf commented 4 years ago

hi @mikehelmick thanks for the answer,

There is a lot of European Union countries trying to implement this as soon as possible. We have integration/e2e testing scheduled in about 7-10 days, prod rollout before the end of month AFAIK.

Some countries are building their own key servers, we are trying to use upstream for as long as possible but this might force us to fork it. We are currently evaluating if the fork is needed or not while architecting the new solution.

Any help from you guys would be highly appreciated, it is quite a blocker for us.

Thanks a lot!

mikehelmick commented 4 years ago

I'm don't think we can meet that timeline of having this production ready by the end of the month. The EFGS federation protocol is quite different from the one that we already implement for federation.

If you wanted to contribute code back to this repo to enable that, we would be available to review (it requires assigning copyright to Google and singing our Common Licensing Agreement).

I think that we will need to support this eventually, but have a pretty full schedule in front of us right now.

marekaf commented 4 years ago

I understand that. Honestly I don't understand the current federation implementation in key server - can't find proper docs for that. I'll try to dig into the code to see if we can reuse some of it or, as you say, reimplement it for EFGS.

At least I know now for sure that it is not compatible with EFGS, not sure why they decided to go 'their way'.

I didn't expect you could re-prioritize this but I'm glad to hear that this is probably going to be supported eventually. Please let me know if there's any ETA on that.

Thanks!

mikehelmick commented 4 years ago

Note: The EFGS protocol is not compatible w/ v1.5 level exports and revised keys.

mikehelmick commented 4 years ago

update - I'm have done a full review of the EFGS protocol and implementation.

I'm in communication with that team and working out what a compatibility model would look like

github-actions[bot] commented 4 years ago

This issue is stale because it has been open for 14 days with no activity. It will automatically close after 7 more days of inactivity.

sherifkozman commented 4 years ago

@mikehelmick I see an open PR on the EFGS that is in review, any updates on that front?

mikehelmick commented 4 years ago

coding is in progress

mikehelmick commented 4 years ago

/assign @whaught

Work tracking (list not currently complete

[ ] efgs-upload model / config
[ ] traveler fanout config
[ ] efgs-upload job
[ ] efgs-upload scheduler (terraform)
[ ] efgs-download model/config
[ ] efgs-download job
[ ] efgs-download scheduler (Terraform)

mikehelmick commented 4 years ago

Thinking about this a bit more - if we upload only valid keys ( and not export padding), the EFGS re-publishing will leak our padding. I think we need

persist the export padding to the database
publish the same keys that are present in export files on a similar cadence as the upload batches

mikehelmick commented 3 years ago

Quick update for the community.

This work is currently on hold as we assess global roaming solutions.

mikehelmick commented 3 years ago

/close

We are not able to proceed with this work at this time. We will revisit in the future if this work becomes feasible.

google-oss-robot commented 3 years ago

@mikehelmick: Closing this issue.

In response to [this](https://github.com/google/exposure-notifications-server/issues/965#issuecomment-723175883): >/close > >We are not able to proceed with this work at this time. We will revisit in the future if this work becomes feasible. Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.