Add guideline for how to get started when using data in CSV files

OpenSenseAction / poligrain

Simplify common tasks for working with point, line and gridded sensor data, focusing on rainfall observations.

https://poligrain.readthedocs.io

BSD 3-Clause "New" or "Revised" License

2 stars 10 forks source link

Add guideline for how to get started when using data in CSV files #34

Closed cchwala closed 1 week ago

cchwala commented 3 months ago

Since many people start with CSV data and since the process to get a xarray.Dataset is a bit complex if doing this for the first time, we should have an example notebook which gives a step-by-step guide.

We can maybe use a variation of this code from the sandbox to show the relevant use of xarray.

cchwala commented 3 months ago

Up for discussion:

Should we already resample with pandas before building the xarray.Dataset?
Anyway we need to align the time dimension before building the xarray.Dataset, e.g. because (some) PWS data will come at more-or-less equidistant time stamps at 5-minute plus some seconds.

cchwala commented 3 months ago

@fenclmar The work to solve this issue, is strongly related to the planed WG1 work on on conversion of datasets, in case they stem from CSV files. Hence, it might make sense to first solve this issue and then build on top of that.

cchwala commented 3 weeks ago

Based on discussion (during video call today) we do the following:

add example notebook that presents a kind of recipe how to get from CSV data to xarray.Dataset in correct format
do it for PWS first based on trainging school preparation notebook
- adjust existing code to use several individual CSV files for individual station (e.g. ten stations for short period)
- add example code for handling of duplicate coordinates and missing time steps using xarray and/or pandas, related to #52
(later add approach for CML data from CSV, could be done e.g. with E-Band data from CZ)