airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.91k stars 4.09k forks source link

[Source-QuickBooks] Getting Error Trying to Setup a Source ('str' object has no attribute 'eval') #46579

Open Omid-Archie opened 1 week ago

Omid-Archie commented 1 week ago

Connector Name

Source-QuickBooks

Connector Version

3.0.25

What step the error happened?

Configuring a new connector

Relevant information

Configuration check failed "Unable to connect to stream accounts - 'str' object has no attribute 'eval'". I get this error with OpenId/Accounting token. I can provide the server information in private.

Relevant log output

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/checks/check_stream.py", line 42, in check_connection
    stream_is_available, reason = availability_strategy.check_availability(stream, logger, source)
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/streams/http/availability_strategy.py", line 50, in check_availability
    get_first_record_for_slice(stream, stream_slice)
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/streams/utils/stream_helper.py", line 40, in get_first_record_for_slice
    return next(records_for_slice)
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/declarative_stream.py", line 126, in read_records
    yield from self.retriever.read_records(self.get_json_schema(), stream_slice)
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/retrievers/simple_retriever.py", line 339, in read_records
    self.cursor.close_slice(_slice, most_recent_record_from_slice)
  File "/airbyte/integration_code/source_quickbooks/components.py", line 72, in close_slice
    most_recent_record=LastRecordDictProxy(most_recent_record, {self.cursor_field.eval(self.config): "MetaData/LastUpdatedTime"}),
AttributeError: 'str' object has no attribute 'eval'
2024-10-08 14:24:37 ERROR i.a.w.i.VersionedAirbyteStreamFactory(internalLog):304 - Check failed
2024-10-08 14:24:37 INFO i.a.c.ConnectorMessageProcessor(updateConfigFromControlMessage):231 - Checking for optional control message...
2024-10-08 14:24:37 INFO i.a.c.ConnectorMessageProcessor(updateConfigFromControlMessage):253 - Optional control message not found. Skipping...
2024-10-08 14:24:37 INFO i.a.c.ConnectorWatcher(saveConnectorOutput):162 - Writing output of cf9c4355-b171-4477-8f2d-6c5cc5fc8b7e_5f8646bd-3f9f-474f-85a3-e6ce25be75ed_0_check to the doc store
2024-10-08 14:24:37 INFO i.a.c.ConnectorWatcher(markWorkloadSuccess):167 - Marking workload cf9c4355-b171-4477-8f2d-6c5cc5fc8b7e_5f8646bd-3f9f-474f-85a3-e6ce25be75ed_0_check as successful
2024-10-08 14:24:37 INFO i.a.c.ConnectorWatcher(exitProperly):215 - Deliberately exiting process with code 0.
2024-10-08 14:24:37 INFO i.a.c.i.LineGobbler(voidCall):166 -

Contribute

natikgadzhi commented 1 week ago

@brianjlai I have a sneaky suspicion that one of the recent availability strategy cleanups caused a regression in the custom component. Who would be the right person to take a look here?

brianjlai commented 1 week ago

@natikgadzhi this doesn't appear to be an availability strategy cleanup specific thing. I can see why you might think that because availability strategy is invoked, but the root cause looks like we're not instantiating the cursor_field as an InterpolatedString (which has an eval method) and instead are just instantiating it as a regular string object. Even if availability strategy was removed out of the code, this would probably end up failing mid-way through the sync due to the field not getting instantiated correctly.

I would have always expected this to be an InterpolatedString as per this: https://github.com/airbytehq/airbyte/blob/2fa35ab30bfe80be6b3ef4763e5e540101724462/airbyte-cdk/python/airbyte_cdk/sources/declarative/incremental/datetime_based_cursor.py#L92 . But this needs more investigation into why we're not constructing the DatetimeBasedCursor instance correctly

natikgadzhi commented 1 week ago

There are a few issues in Quickbooks that @pabloescoder recently worked on. His work solves a few other things and it is NOT merged yet, blocked by regression tests runners crapping out.

We should team up with @alafanechere and @pabloescoder as the first collaboration of tooling <> community eng, and make regression tests play nice, unblock previous PRs, and then ask Dhroov to fix this one as well.

Omid-Archie commented 4 days ago

@natikgadzhi @brianjlai Hi guys, any updates on this? I need it for my corp ASAP.

vivek-immersa commented 2 days ago

Yeah. I would love to get an update on this as well as our syncs have been failing for 2 days now with the same issue OP mentioned