search

ropensci / rgbif

Interface to the Global Biodiversity Information Facility API
https://docs.ropensci.org/rgbif
Other
155 stars 50 forks source link

Incorporate Flatterer for Frictionless DP #759

Open CecSve opened 1 week ago

CecSve commented 1 week ago

It might be worthwhile to explore whether Flatterer could be a useful tool for rgbif?

With Flatterer you can:

  • easily convert JSON to flat relational data such as CSV, XLSX, Database Tables, Pandas Dataframes and Parquet;
  • convert JSON into data packages, so you can use Frictionless data to convert into any database format;
  • create a data dictionary that contains metadata about the conversion, including fields contained in the dataset, to help you understand the data you are looking at;
  • create a new table for each one-to-many relationship, alongside _link fields that help to join the data together.
jhnwllr commented 4 days ago

rgbif already does a fairly good job of giving back flat tables using jsonlite.

https://github.com/jeroen/jsonlite

I think that flatter could perhaps be more useful for pygbif, where we don't give back flat tables.