Schema updates - Githubissues

peterdesmet commented 1 year ago

[x] Fix #39 by making NaN a missing value
[x] Fix #40 by making a number of columns optional
[x] Switch order to lat, long
[x] Add source_file column, #41

Todo

[x] Verify with @adokter
[x] Update https://aloftdata.eu/vpts-csv/#example with new column and data s3://noaa-nexrad-level2/2016/09/01/KBGM/KBGM20160901_000212_V06
[x] Indicate that sort order is radar, datetime, height and source_file. The latter will make sure that duplicates are more easy to spot within a height
[x] Move source file to aloft repo
[x] Ping @stijnvanhoey about these changes

peterdesmet commented 1 year ago

@stijnvanhoey these changes can be implemented.

peterdesmet commented 1 year ago

I've added a requirement to source_file inspired by Frictionless's url-or-path that restricts values that start with ../ and /. This to avoid injection of file paths that might be harmful. I hope that is ok.

peterdesmet commented 1 year ago

https://aloftdata.eu/vpts-csv/ now reflects the updated format

stijnvanhoey commented 1 year ago

@peterdesmet can we reflect the version of the csv-schema in a follow up PR? cfr. https://github.com/enram/vptstools/issues/21

peterdesmet commented 1 year ago

Done, there is now a v1.0 release: https://github.com/enram/vpts-csv/releases/tag/v1.0

enram / vpts-csv

Schema updates #42