Add I/O streaming to LROSE tools

NCAR / lrose-core

Core C/C++ code for LROSE.

https://www.eol.ucar.edu/content/lidar-radar-open-software-environment

Other

92 stars 51 forks source link

Add I/O streaming to LROSE tools #67

Closed leavesntwigs closed 4 years ago

leavesntwigs commented 5 years ago

It would be nice to avoid writing intermediate data files and instead stream the data from one LROSE tool into the next tool. For example, Linux piping would be nice, particularly for large data sets.

nguy commented 4 years ago

I agree that this would be a nice feature

leavesntwigs commented 4 years ago

This data pipeline philosophy will probably be implemented in the new Science Gateway that is in development. The intermediate data file management will be hidden to the user, but available if needed.