Pan-Canada Immunization Data on Federated Data Architecture

[nghia71]

Pan-Canada Immunization Data on Federated Data Architecture -- A Data Mesh Reference Implementation

Objectives:

• Distributed computational data governance on federated data architecture
• Request based federated data analytics
• Population-level analytics on continuous event streams

Deadline:

• Dec 8 (all technical)
• Dec 11-21 (presentations, talkshow, etc)

Short descriptions:

• Generating a set of synthetic patient population - EHRs with detailed events (encounters, prescriptions, procedures, vaccinations, etc) fully respecting Canadian demographics (population, location, gender, race, living conditions, etc)
• Establishing 13 computing clusters on GCP, each for a PT jurisdiction
  • Each cluster hosts its EHRs in some HAPI FHIR server(s)
  • A local analytics service platform:
    • Data Governance Gateway (DGG):
      • configured with a set of machine-readable data standards for ESG
      • configured with a set of machine-executable data governance policy for ACG
      • monitoring system status, track lineage, and audit operations
    • Event Streaming Gateway (ESG):
      • turn EHR vaccination events into streams based on instructions from DGG
      • perform in-stream processing instructions (querying, aggregation, filtering)
    • Local Analytics Pipeline (LAP):
      • receiving feredrated data analytics request, performing request, produces results.
    • Access Control Gateway (ACG):
      • receives request via a federated data analytics protocol/API, execute data governance policy on the request, forward it to FAP if compliant,
      • receives response from FAP, execute data governance policy on the response, forward it to the requestor.
• Establishing 1 computing cluster on GCP for federal
  • A federated analytics service platform:
    • Data Governance Gateway (DGG):
      • configured with a set of federated data analytics requests
      • configured with a set of population-level event stream analytics
      • monitoring system status, track lineage, and audit operations
    • Federated Analytics Pipeline (FAP):
      • creates feredrated data analytics request, delivers requests to PT, aggregates and produces results.
      • creates queries for vaccination with Adverse Effects, performs analytics, and produces results

Platform:

• GCP
• Github, CI/CD
• 14 secure clusters.
• Access from public internet
• Showcase with real interactive data on UI.

People:

• Luc, Nghia
• somebody for data engineer (Kafka, security, FHIR, database, etc)
• somebody for UI (multiple dashboards, interactive UIs, time series, etc)
• somebody for presentation deck, flyer, (web)site design (look/feel/UX)
• somebody to write and tell stories
• somebody to manage schedule for all the activities.