search

enram / vpts-csv

Data exchange format for biological signals detected by weather radars
https://aloftdata.eu/vpts-csv/
MIT License
3 stars 3 forks source link

Questions about the temporal resolution #10

Closed niconoe closed 3 years ago

niconoe commented 3 years ago

Currently, vol2bird CSV (that we take inspiration from)

1) Do we want to enforce/specify a given temporal resolution (fixed, min and/or max) 2) Do we want to allow gaps (example: data interval is 5 minutes, but missing data between 2 and 4 pm) 3) Do we want to express the resolution in metadata (or let the consumers infer it from the content of the CSV file). If yes, how? 4) If there are gaps, do we want to express that fact in metadata? how?

Additional considerations:

5) Vol2bird CSV output currently express time as "HHMM", which doesn't allow to go under a minute

Personal opinions:

1) not a fixed one (too restrictive), min/max: maybe (but I don't see a very good reason yet) 2) I don't know. @peterdesmet / @adokter : would we dismiss a lot of user and use-cases by enforcing that? 3) I'd say yes. Maybe a simple integer (number of seconds) is good enough 4) I think yes 5) I was wondering if replacing (compared to vol2bird text output) the date/time columns by a proper ISO 8601 datetime wouldn't be a nice solution (easy to parse in any language thanks to libraries, allows expressing time zones, allows the temporal resolution to be 1 sec, ...)

peterdesmet commented 3 years ago
  1. Temporal range can be whatever, but unlikely to have use cases longer than a month. It does need to be a single radar though
  2. Gaps should be allowed (very common)
  3. Yes, single integer with resolution, not considering gaps
  4. ~Could be good, but not sure how without making it verbose. Maybe list of yyyy-mm-dd-hh number of data points?~ This was discussed with the group, too variable. Users will need to look at the data.
  5. Yes, ISO please
niconoe commented 3 years ago

Thanks, decisions noted in the draft specifications document.