The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
For a specific customer account we were using, they were not seeing the Deals stream.
After investigating we found that the fields meta data coming back from the Deals stream was missing the "json_type" and "length" property in some of the data. This was causing the deals stream to not be added to the list of streams in the Zoho Streams Factory.
"json_type" is a required type thus this makes sense but "length" is not required thus it shouldn't cause an exit on that.
Upon further inspection we found that even though length is set to Optional, it was not being defaulted to None thus was actually a required field according to this stackoverflow post https://stackoverflow.com/questions/70809438/python-dataclasses-with-optional-attributes.
The reason the "json_type" field was missing is unknown, it would be an issue on Zoho's API. But we found upgrading the api version in the connector from v2 to v4 fixes this problem and all the required fields are present.
Thus this simply requires 2 small fixes:
Updating v2 to v4 for the api version being used in the ZohoCRM connector
Have the optional fields in the FieldMeta class default to None
Connector Name
source-zoho-crm
Connector Version
0.1.2
What step the error happened?
None
Revelant information
For a specific customer account we were using, they were not seeing the Deals stream. After investigating we found that the fields meta data coming back from the Deals stream was missing the "json_type" and "length" property in some of the data. This was causing the deals stream to not be added to the list of streams in the Zoho Streams Factory. "json_type" is a required type thus this makes sense but "length" is not required thus it shouldn't cause an exit on that. Upon further inspection we found that even though length is set to Optional, it was not being defaulted to None thus was actually a required field according to this stackoverflow post https://stackoverflow.com/questions/70809438/python-dataclasses-with-optional-attributes.
The reason the "json_type" field was missing is unknown, it would be an issue on Zoho's API. But we found upgrading the api version in the connector from v2 to v4 fixes this problem and all the required fields are present.
Thus this simply requires 2 small fixes:
I will post a PR for this issue shortly.
Relevant log output
No response
Contribute
PR for the fix found here: https://github.com/airbytehq/airbyte/pull/27423