neherlab / covid19_scenarios

Models of COVID-19 outbreak trajectories and hospital demand
https://covid19-scenarios.org
MIT License
1.36k stars 354 forks source link

Document, improve schema modification workflow #767

Open ivan-aksamentov opened 4 years ago

ivan-aksamentov commented 4 years ago

πŸ™‹ Feature Request

πŸ”¦ Context

The data flowing through application can sometimes change, depending on new scientific ideas and the epidemiological situation. We value backward compatibility with the old file and URL formats, but changing the data schema and adapting I/O serialization code for the new format has proven to be a difficult task. It requires many steps. Many I/O-related pieces of the application depends on it being done correctly.

😯 Describe the feature

We need to thoroughly document the steps that need to be taken in order to migrate the application to the new version of the schema.

Some basics can be found here: https://github.com/neherlab/covid19_scenarios/pull/757#issuecomment-661027281

The best place is probably in the readme, in (or after) the "Schemas" section https://github.com/neherlab/covid19_scenarios/blob/9469b995be/README.md#schemas

We might also consider improving the entire workflow and reorganizing the code to make migrations more convenient and less error prone.