What is the data source: Affinity REST API; Mixture of V2 (OpenAPI-based, has paging, accumulation, etc. but incomplete as of 2024-11-19) and V1 (Complete API, but no accumulation, meaning hundreds of requests needed to complete some entities; Access to some entities (like Notes) which are not (yet) available in V2).
Current Status
[x] I plan to write it
[ ] I've written it and run it for fun
[ ] I run it on my local laptop
[ ] I run it in production
I tried following destinations with it: [e.g. duckdb, bigquery]
What source does/will do
Please provide descriptions up to 3 most important use cases that users of this source do. Those use cases will be:
pull entities (companies, persons, opportunities) from the CRM
augment each entity with its
fields
notes
transform the fields (fields can have different value types, ranging from simple text, number to lists of complex data objects) of these entities in a safe way into the target format
Test account / test data
[ ] I'd like dltHub to create a test account before we start
[ ] I have test account and can share my test data or a test account with dltHub after we merge the source
^ Given the price tag of Affinity, it is unlikely that we will be able to get a demo account for testing. I can reach out to our key account manager and see what they say, though. if they say no, I'd suggest to store local fixtures of the API requests and replay them in the test environment. Please let me know if this is a dealbreaker.
Additional context
Most similar existing verified source is HubSpot.
The API V2, which is needed, really, to implement this, is currently only available for customers with an enterprise plan.
There is an OpenAPI spec available, which we've successfully used to generate a client. We're currently using said client in a custom pipeline to extract all data from Affinity and load it into Snowflake. There is currently no maintained Python-based client, but it might be worthwile to see if an OpenAPI-based client can be generated easily using openapi-generator. The first test will most likely be to use the API directly.
Rate-limiting is quite strict with both V1 and V2 of the Affinity API. The source will need to implement throttling like the Workable of Freshdesk sources and retries like the Google Sheets source.
Quick source info
Current Status
What source does/will do
Please provide descriptions up to 3 most important use cases that users of this source do. Those use cases will be:
Test account / test data
dltHubto create a test account before we startdltHubafter we merge the source^ Given the price tag of Affinity, it is unlikely that we will be able to get a demo account for testing. I can reach out to our key account manager and see what they say, though. if they say no, I'd suggest to store local fixtures of the API requests and replay them in the test environment. Please let me know if this is a dealbreaker.
Additional context
HubSpot.WorkableofFreshdesksources and retries like theGoogle Sheetssource.