search

CDCgov / nis-py-api

Python API to NIS data
Apache License 2.0
1 stars 0 forks source link

Unify python Socrata Open Data API tooling? #46

Open dylanhmorris opened 3 days ago

dylanhmorris commented 3 days ago

I happened to notice some SODA query/API helpers here @swo. Since this is a common need across CFA (there are R helpers within forecasttools and @AFg6K7h4fhy2 is thinking about recreating them in forecasttools-py ), we should discuss how not to duplicate effort and create robust tools.

sodapy exists, but unfortunately it is no longer maintained. I see one attempt to fork it and continue development: sodapy2.

We could:

  1. Build our own very lightweight helper package on CDCGov
  2. Fork the original sodapy for ourselves
  3. Contribute to sodapy2.
AFg6K7h4fhy2 commented 3 days ago

Explicitly stating agreement with above DHM comment.

Conversation here seem like it could easily reduce wasted motion.

My preference is (1) but this might change were I less ignorant concerning sodapy and sodapy2 internals.

swo commented 5 hours ago

I like option 1.

Now that you're here asking about this, I prefer option 1 to option zero (status quo). I've written Socrata queries by hand a few times and re-implemented this kind of interface at least once. Pagination is new but I think that's important.

sodapy was a pretty simple set of wrappers around requests. It included support for things I don't know if we need now (e.g., uploading data) and also made some API choices that I think were suboptimal (e.g., get_all() iterates over the records one by one). I don't know that there's a lot there we need to rescue, which makes options 1 and 2 less appealing.