Open laumc98 opened 4 months ago
Note: Pipedrive connector needs a bit of cleanup, this PR should get most of the boilerplate out of the way: https://github.com/airbytehq/airbyte/pull/37224
Once that's merged, we should take a look at how the additional schema fields are populated.
@laumc98, is it that absolutely none of the dynamic fields are getting synced, or only the ones with long names? Can you share examples of fields that worked and field names that don't show up?
Posted in another issue (#9433), but as that is closed, I'd better post it here again: https://github.com/airbytehq/airbyte/issues/9433#issuecomment-2196962520
Each deal consists of standard data fields but can also contain a number of custom fields. The custom fields can be recognized by long hashes as keys. These hashes can be mapped against DealField.key. The corresponding label for each such custom field can be obtained from DealField.name.
These custom fields are missing from both the normalized tables and the raw JSON data in airbyte_internal. It would be great if this gets addressed.
My recommended solution would be to include a column custom_fields in the normalized output which includes the custom field data as JSON. To my understanding this would be the most flexible solution.
@neurotroph thank you for pinging us with this again. We should get to it, but Pipedrive is one of the marketplace connectors, and we don't quite have the capacity to work on it right away.
I hope we can get to this in a couple months after we get all connectors on more standardized tooling.
In the meantime, if anyone has an implementation for custom fields in their connector builder and wants to copy it here, I'm happy to try and push it in
Connector Name
source-pipedrive
Connector Version
2.2.2
What step the error happened?
During the sync
Relevant information
I've set a connection between Pipedrive and Redshift, but there are several deal custom fields (the ones with long hashes as keys) that are not being synchronized to my warehouse. I've already looked at the raw data, and the fields are not there either.
Relevant log output
No response
Contribute