STATE OF THE PROJECT

[jeanpaulrsoucy]

This issue should be used for tracking and discussing the overall state of the project, FAIR COVID-19 Data for 🇨🇦.

Project goal

The goal of this project is to FAIRify the data stored in the Canadian COVID-19 Data Archive.

Slide deck

Need to create an evolving "State of the Project" slide deck (e.g., using HackMD).

Progress to date

The Canadian COVID-19 Data Archive is a publicly accessible repository storing daily snapshots of hundreds of datasets, web pages and reports related to the COVID-19 pandemic in Canada, operating since August 2020. Without the Archive, many of these datasets would already be lost. As of February 2022, the Archive contains over 160 GB of data spanning about 600 datasets. Data collection is automated via a custom Python package (archivist) and a set of GitHub actions Covid19CanadaBot. All data are available via a publicly exposed S3 bucket, documented in a data catalogue on GitHub and searchable using a basic data explorer.

The Canadian COVID-19 Data Archive is also being used to support the ongoing maintenance of the COVID-19 Canada Open Data Working Group's two datasets: Covid19Canada ("Epidemiological Data from the COVID-19 Outbreak in Canada") and CovidTimelineCanada ("Timeline of COVID-19 in Canada").

Tasks in progress

Data tool and API

The main output of this project will be a publicly accessible data tool allowing users to find, manipulate and export relevant Canadian COVID-19 data in a common, FAIR format. The simple front-end tool (#8) will be supported by a robust, well-documented API (#9) for more advanced users.

Rich metadata

The datasets available in the Archive must be supported by rich metadata allowing users to find, understand and apply each of the derived datasets (#7, #13, #4), including understanding data licenses (#5, #6).

Workflows for creating FAIR data

We will need to write custom workflows to process each dataset (#15) into a common, FAIR data format (#10).

A more sustainable process

Another goal of the project is to be sustainable, which begins with improving the sustainability of the Canadian COVID-19 Data Archive, both in terms of maintaining the automation (#2), list of datasets (#4) and the underlying infrastructure (#3).

[colliand]

While reviewing the issues, I am reminded that it is wise to put socks on before putting on shoes. I suggest that the anchor comment above be enriched with a phasing of main tasks. A GANTT chart (built with the newly available Mermaid.js offered by GitHub) may be helpful. To the extent possible, I suggest we strive to address the transitions between phases to allow for some work on later stages in our project to start while work is underway on earlier stages. Making this work will likely require some waterfall-style specifications to define the glue that links the components together. You can knit socks in the evening while working on shoes in the shop during the day...

[colliand]

I enjoyed a quick Zoom meeting today with @stevencarlislewalker. We discussed this project and the CANMOD digitization project that builds on the IIDA project led by @davidearn. Steve and I recognized exciting interconnections between these projects and landed on a proposed path forward:

• Meet next week to explore synergies. Draft agenda for the meeting
  1. introductions
  2. brief overview of IIDA/Digitization (David?)
  3. screen share tour of IIDA/Digitization repos (Steve)
  4. screen share tour of CCODWG resources (Jean-Paul)
  5. discussion: next steps?
  6. (if time allows). discuss PIMS-hosted event
• Organize PIMS-hosted event (May 2022?)
  1. Jim Colliander was awarded $5K + logistical support to host a "kickoff event" for the FAIRification project
  2. Colliander had recent conversations about the FAIRification and Digitization projects with
     • David Castle (former VPR at UVIC; now working in Office of Chief Science Advisor Mona Nemer; Chair of Scientific Committee of the World Data System)
     • Karen Payne (Executive Director of the International Technology Office of the World Data System; ITO was recently established at UVIC with funding from the Digital Research Alliance of Canada "the Alliance")
     • Mark Leggott (Director International Relations and Research Data Management for the Alliance)
     • Jayadev Athreya (PIMS Interim Director)
  3. The PIMS event can be convened to:
     • Showcase Digitization/IIDA
     • Showcase work of CCODWG
     • Catalyze partnerships with Alliance (data hosting + compute) and WDS (international collaborators)
  4. PIMS and partners are organizing PRIMA2022 in December 2022. This meeting happens every 4 years and brings together mathematicians across the Pacific Rim. There will be a special session on mathematics of COVID. PRIMA presents an opportunity to establish collaborations with pacific rim partners.
• Sustainability?
  1. Alliance could (should!) emerge as a compute + data-hosting provider to prepare Canada for the next pandemic
  2. Canada should generously lead data + compute + modeling efforts globally using an open science strategy
     • Lead by example?
     • The COVID-19 pandemic is a top-of-mind example of how science + data can serve the public good.
     • An effective unification of efforts to prepare for the next pandemic within CANMOD, within Canada, and involving international partners can hopefully catalyze similar unified approaches to the climate crisis, problems with misinformation/disinformation, and other grand challenges.