NEONScience / NEON-utilities

Utilities and scripts for working with NEON data. Currently: an R package with functions to join (stack) the month-by-site files in downloaded NEON data, to convert data to geoCSV format, and to download data from the API.
GNU Affero General Public License v3.0
57 stars 36 forks source link

Provide rda files in cross-language format #126

Closed ethanwhite closed 1 year ago

ethanwhite commented 1 year ago

Is your feature request related to a problem? Please describe. Some of the rda files in this repo are useful to other projects (e.g., we're using table_types.rda in https://github.com/weecology/neonwranglerpy/), but rda files are difficult to work with in languages other than R.

Describe the solution you'd like Either in this repo or elsewhere provide those same files in a format that is accessible across languages (e.g., csv, arrow). If they are already available in a different format elsewhere just point me in the right direction.

Describe alternatives you've considered Doing the conversion from rda ourselves - but this requires R (at least for Python) which means doing so in Python packages introduces a difficult external dependency. We could automate this conversion and serve the more compatible formats ourselves if doing so is out of scope for this repo and the files aren't available elsewhere.

cklunch commented 1 year ago

@ethanwhite Thanks for the suggestion! This is an easy add, and easy to keep updated with new versions of the package. I should be able to do this by the end of the week.

For the table_types table specifically, you might be interested in the findTablesByFormat() function, which infers the table types based on the formatting of the published file names. We use it to bridge the time between neonUtilities versions, so if new data tables or data products are added between versions, the stacking functions don't stop working.

It's also great to know about neonwranglerpy! Of course we've talked a lot internally about developing a Python version of neonUtilities, but we haven't had the resource availability to do it so far. We'll definitely get in touch with you if that changes in the future, and in the meantime, let us know if there are other ways we can make your development easier.

ethanwhite commented 1 year ago

@cklunch Awesome! Thanks! Appreciate the pointer and definitely do let us know if you all start working on a Python port and we'll be happy to help with testing, etc.

cklunch commented 1 year ago

@ethanwhite The files that are packaged with neonUtilities are now duplicated in a helper_files folder in this repo, with a short readme. https://github.com/NEONScience/NEON-utilities/tree/main/helper_files

Feel free to re-open this issue if this doesn't meet your needs!

ethanwhite commented 1 year ago

Thanks! This looks perfect!