search

iobis / pyobis

OBIS Python client
https://iobis.github.io/pyobis
MIT License
15 stars 10 forks source link

Add capability to download Full OBIS Export of presence records? #94

Open MathewBiddle opened 2 years ago

MathewBiddle commented 2 years ago

OBIS provides a full export of presence records [1] in both csv [2] and parquet [3]. Would it be feasible to add something to pyobis that goes out and efficiently grabs the whole export?

I'm sure folks smarter than me know of ways to parallelize the call or do something fancy with the available services to download in a reasonable time frame.

[1] - https://obis.org/data/access/ [2] - https://obis-datasets.ams3.digitaloceanspaces.com/exports/obis_20220710.csv.zip [3] - https://obis-datasets.ams3.digitaloceanspaces.com/exports/obis_20220710.parquet

Could also be a waste of time for this package.

7yl4r commented 2 years ago

cough cough bittorrent cough. :zipper_mouth_face: