Llamabagel / transpo-shared

Shared components used by Route 613
MIT License
0 stars 0 forks source link

Data Versioning #2

Open dellisd opened 5 years ago

dellisd commented 5 years ago

The GTFS data provided by OC Transpo changes periodically and an efficient way of versioning the packaged data will be needed to allow the client to always be served using the latest data. Separately, a way to version the layout of the data packages will be required in case the data format/contents are modified in the future.

Data Versioning

Here we can just use a date stamp to identify the day on which the data package was published (i.e. YYYYMMDD). In case multiple packages need to be published on the same day, we can use a decimal point hyphen and another number to indicate the additional version number (similar to semantic versioning).

For example, if 20190315 is published but there is an error that is caught immediately and needs to be corrected, a new package can be published as 20190315-1, and then 20190315-2, ..., 20190315-13, etc.

This gives each package an easily readable identifier.

Schema Versinoning

Because the data requirements of the clients may change over time, a separate version will need to be considered for the data schema. This can just be an integer starting at "1" and incremented over time as needed.

In practice

When the client requests a copy of the latest data, it will send the server the current version of data stored on the client as well as the version of schema that it supports. The server will then get the latest copy of data for that schema and return it.

dellisd commented 5 years ago

Changed additional version modifier to - separator instead of . for easier parsing from file extension.