search

move-coop / parsons

A python library of connectors for the progressive community.
https://www.parsonsproject.org/
Other
257 stars 125 forks source link

New connector for working with the Catalist Match API #912

Closed austinweisgrau closed 8 months ago

austinweisgrau commented 8 months ago

Connector for working with the Catalist Match API.

The Catalist Match tool requires OAuth2.0 client credentials for the API as well as credentials for accessing the Catalist sftp bucket. Each Catalist client is given their own bucket alias named after a tree species, used for constructing the filepath within the sftp bucket.

Accessing the Catalist sftp bucket and Match API both require the source IP address to be explicitly white-listed by Catalist.

Example usage:

tbl = Table.from_csv(...)
client = CatalistMatch(...)
match_result = client.match(tbl)

Tests

Tests are included in this PR.

Changes to parsons

Two changes are made to the parsons package outside of this new connector.

28949a9a20fe4422aae3b0ac6373e912c61d4b4c makes a small change to the api_connector error handler to look for error messages in a response text attribute, where the error message is found in responses from the Catalist API.

1ffb04631bc79e99bbd523bdf314ba179849104a adds a development dependency pytest-mock to support the ergonomic use of mocking in pytest. This dependency is then used in the tests on the catalist match connector in 31f3a1816cdb09d5004fb11f70d1bd6da0182cd1.

Expected future changes

Following a community development session, JSON responses from the API will be parsed using python dataclasses. Currently JSON responses are returned raw, which matches the normal pattern across parsons connectors.

austinweisgrau commented 8 months ago

Working on updates to type hints to be backwards compatible with Python versions back to 3.8

austinweisgrau commented 8 months ago

@IanRFerguson or @shaunagm Do you know what's up with that docker failure? When I hit details I just get {"error": "Our service is temporarily unavailable. We'll be back soon"}

austinweisgrau commented 8 months ago

@IanRFerguson or @shaunagm Do you know what's up with that docker failure? When I hit details I just get {"error": "Our service is temporarily unavailable. We'll be back soon"}

NVM I got it with 001b527

austinweisgrau commented 8 months ago

Looks excellent overall - you crushed this! I have a few comments throughout but nothing structural or functional, the code is super legible and easy to navigate.

One note - I think adding a docs/catalist.rst file is required before merging. I left a comment on setup / obtaining client creds and I reckon the docs would be a good place to outline that

Added the docs/catalist.rst file with 9b735233c193be56535e14fec0c137a97ab88a37, that was new for me though so not totally sure I did that correctly.