Note: great showcase of official statistics data hosted as parquet files

rOpenSpain / spanishoddata

Access national high-quality and open-access datasets on movement patterns derived from mobile telephone datasets

https://ropenspain.github.io/spanishoddata/

Other

24 stars 0 forks source link

Note: great showcase of official statistics data hosted as parquet files #10

Open e-kotov opened 4 months ago

e-kotov commented 4 months ago

https://observablehq.com/@fil/insee-parquet

This is a great showcase and argument for advocating to re-host the origin-destination data as parquet.

Meanwhile, I tested a few data repositories, such as Zenodo, Harvard Dataverse, OSF, Figshare. None provide a good way to access a parquet file. One has to use their API and download the whole file to work with it. Does not seem to be possible to query them.

Robinlovelace commented 4 months ago

Very interesting, seems like Amazon S3 is a bit of a standard, I wonder how much hosting would cost. We could add some mechanism to limit the rate e.g. API key/password.