airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.16k stars 4.13k forks source link

Source ZohoCRM: API Version is out of date and the optional fields are not defaulted to None #27422

Closed richardlam-quotapath closed 4 months ago

richardlam-quotapath commented 1 year ago

Connector Name

source-zoho-crm

Connector Version

0.1.2

What step the error happened?

None

Revelant information

For a specific customer account we were using, they were not seeing the Deals stream. After investigating we found that the fields meta data coming back from the Deals stream was missing the "json_type" and "length" property in some of the data. This was causing the deals stream to not be added to the list of streams in the Zoho Streams Factory. "json_type" is a required type thus this makes sense but "length" is not required thus it shouldn't cause an exit on that. Upon further inspection we found that even though length is set to Optional, it was not being defaulted to None thus was actually a required field according to this stackoverflow post https://stackoverflow.com/questions/70809438/python-dataclasses-with-optional-attributes.

The reason the "json_type" field was missing is unknown, it would be an issue on Zoho's API. But we found upgrading the api version in the connector from v2 to v4 fixes this problem and all the required fields are present.

Thus this simply requires 2 small fixes:

  1. Updating v2 to v4 for the api version being used in the ZohoCRM connector
  2. Have the optional fields in the FieldMeta class default to None

I will post a PR for this issue shortly.

Relevant log output

No response

Contribute

PR for the fix found here: https://github.com/airbytehq/airbyte/pull/27423

pabloescoder commented 1 year ago

I want to work on this issue, can someone please assign it to me?

justinbchau commented 1 year ago

done ☺️ @pabloescoder