Consider using a CSV-focused data-packaging format

ODM2 / YODA-Tools

Tools to validate and manage YODA files

BSD 3-Clause "New" or "Revised" License

0 stars 1 forks source link

Consider using a CSV-focused data-packaging format #33

Open jkreft-usgs opened 6 years ago

jkreft-usgs commented 6 years ago

YODA and the Excel format are both unfriendly to large datasets, Excel because of row limits, and Yoda because it appears to require loading the entire dataset into memory before being able to generate the YODA file. Supporting a CSV-focused packaging approach such as http://frictionlessdata.io/data-packages/ could make for a large-data-friendly ODM2 data format.

horsburgh commented 6 years ago

@jkreft-usgs - thanks for this. I'm excited to have a look.

emiliom commented 6 years ago

@jkreft-usgs I ran into your comments again just now and bothered to look into Data Packages. I wasn't aware of that container/format/effort. It looks very cool, and exciting -- thanks for the suggestion!