uataq / stilt

Stochastic Time-Inverted Lagrangian Transport model
https://uataq.github.io/stilt/
49 stars 17 forks source link

cross-platform trajectory output #107

Open jmineau opened 2 months ago

jmineau commented 2 months ago

While R is great, and the bindings that @benfasoli wrote for STILT are incredible, for users, like myself, whose language of choice is something other than R, outputting the trajectories to rds format is quite inconvenient. Sure, we could use R conversion libraries like rpy2, but it would be much more convenient if the trajectories could be written to a cross-platform, portable format.

Some ideas include:

jmineau commented 2 months ago

I implemented the option to write to h5 in my PR: #108

benfasoli commented 2 months ago

HDF5 and NetCDF4 are probably already installed since the footprints are NetCDF4 but they aren't very ergonomic for users and usually have clunky install stories because the language bindings are links to the system-level HDF5/NetCDF4 C-lib installs.

Take a look at parquet files for a performant column storage format that works cross-platform without many of the same drawbacks 👍

jmineau commented 2 months ago

Appreciate the insight.

I have implemented parquet as an option and added documentation. I left h5 as an option, but didnt document in the markdown docs. I also added the option and documentation to disable writing trajectories with ''