Note: great showcase of official statistics data hosted as parquet files

Robinlovelace / spanishoddata

Work-in-progress package to get OD data. We may move the repo at some point..

https://robinlovelace.github.io/spanishoddata/

Other

7 stars 0 forks source link

Note: great showcase of official statistics data hosted as parquet files #10

Open e-kotov opened 5 days ago

e-kotov commented 5 days ago

https://observablehq.com/@fil/insee-parquet

This is a great showcase and argument for advocating to re-host the origin-destination data as parquet.

Meanwhile, I tested a few data repositories, such as Zenodo, Harvard Dataverse, OSF, Figshare. None provide a good way to access a parquet file. One has to use their API and download the whole file to work with it. Does not seem to be possible to query them.

Robinlovelace commented 4 days ago

Very interesting, seems like Amazon S3 is a bit of a standard, I wonder how much hosting would cost. We could add some mechanism to limit the rate e.g. API key/password.