search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
16.19k stars 4.14k forks source link

[Source Trustpilot] - Refresh token error - Bad request #42854

Closed btkcodedev closed 3 months ago

btkcodedev commented 3 months ago

Connector Name

source-trustpilot

Connector Version

0.2.1

What step the error happened?

During the sync

Relevant information

Reported by Airbyte User: Reference: https://github.com/airbytehq/airbyte/pull/40267#issuecomment-2255151622

Relevant log output

File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/streams/http/availability_strategy.py", line 56, in check_availability
    is_available, reason = self.handle_http_error(stream, logger, source, error)
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/streams/http/availability_strategy.py", line 85, in handle_http_error
    raise error
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/streams/http/availability_strategy.py", line 50, in check_availability
    get_first_record_for_slice(stream, stream_slice)
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/streams/utils/stream_helper.py", line 40, in get_first_record_for_slice
    return next(records_for_slice)
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/declarative_stream.py", line 128, in read_records
    yield from self.retriever.read_records(self.get_json_schema(), stream_slice)
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/retrievers/simple_retriever.py", line 375, in read_records
    for stream_data in self._read_pages(record_generator, self.state, _slice):
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/retrievers/simple_retriever.py", line 298, in _read_pages
    response = self._fetch_next_page(stream_state, stream_slice, next_page_token)
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/retrievers/simple_retriever.py", line 273, in _fetch_next_page
    return self.requester.send_request(
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/requesters/http_requester.py", line 301, in send_request
    headers=self._request_headers(stream_state, stream_slice, next_page_token, request_headers),
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/requesters/http_requester.py", line 190, in _request_headers
    headers = self._get_request_options(
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/declarative/requesters/http_requester.py", line 174, in _get_request_options
    auth_options_method(),
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/streams/http/requests_native_auth/abstract_oauth.py", line 56, in get_auth_header
    return {"Authorization": f"Bearer {self.get_access_token()}"}
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/streams/http/requests_native_auth/abstract_oauth.py", line 61, in get_access_token
    token, expires_in = self.refresh_access_token()
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/streams/http/requests_native_auth/abstract_oauth.py", line 150, in refresh_access_token
    response_json = self._get_refresh_access_token_response()
  File "/usr/local/lib/python3.10/site-packages/backoff/_sync.py", line 105, in retry
    ret = target(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/airbyte_cdk/sources/streams/http/requests_native_auth/abstract_oauth.py", line 132, in _get_refresh_access_token_response
    response.raise_for_status()
  File "/usr/local/lib/python3.10/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.trustpilot.com/v1/oauth/oauth-business-users-for-applications/accesstoken?response_type=token
2024-07-29 07:20:55 ERROR i.a.w.i.VersionedAirbyteStreamFactory(internalLog):304 - Check failed
2024-07-29 07:20:55 INFO i.a.c.ConnectorMessageProcessor(updateConfigFromControlMessage):231 - Checking for optional control message...
2024-07-29 07:20:55 INFO i.a.c.ConnectorWatcher(run):134 - Writing output of d7e23ea6-d741-4314-9209-a33c91a2e945_15400100_0_check to the doc store
2024-07-29 07:20:55 INFO i.a.c.ConnectorWatcher(run):136 - Marking workload as successful
2024-07-29 07:20:56 INFO i.a.c.ConnectorWatcher(exitProperly):189 - Deliberately exiting process with code 0.
2024-07-29 07:20:56 INFO i.a.c.i.LineGobbler(voidCall):166 - 
2024-07-29 07:20:56 INFO i.a.c.i.LineGobbler(voidCall):166 - ----- END CHECK -----
2024-07-29 07:20:59 platform > Retry State: RetryManager(completeFailureBackoffPolicy=BackoffPolicy(minInterval=PT10S, maxInterval=PT30M, base=3), partialFailureBackoffPolicy=null, successiveCompleteFailureLimit=5, totalCompleteFailureLimit=5, successivePartialFailureLimit=1000, totalPartialFailureLimit=20, successiveCompleteFailures=1, totalCompleteFailures=1, successivePartialFailures=0, totalPartialFailures=0)
 Backoff before next attempt: 10 seconds
2024-07-29 07:20:44 INFO i.a.w.l.c.WorkloadApiClient(claim):75 - Claimed: true for d7e23ea6-d741-4314-9209-a33c91a2e945_15400100_0_check via API for aws-eu-paris-queue-1
2024-07-29 07:20:44 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: CHECK_STATUS — (workloadId = d7e23ea6-d741-4314-9209-a33c91a2e945_15400100_0_check) — (dataplaneId = aws-eu-paris-queue-1)
2024-07-29 07:20:44 INFO i.a.w.l.p.s.CheckStatusStage(applyStage):59 - No pod found running for workload d7e23ea6-d741-4314-9209-a33c91a2e945_15400100_0_check
2024-07-29 07:20:44 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: BUILD — (workloadId = d7e23ea6-d741-4314-9209-a33c91a2e945_15400100_0_check) — (dataplaneId = aws-eu-paris-queue-1)
2024-07-29 07:20:45 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: MUTEX — (workloadId = d7e23ea6-d741-4314-9209-a33c91a2e945_15400100_0_check) — (dataplaneId = aws-eu-paris-queue-1)
2024-07-29 07:20:45 INFO i.a.w.l.p.s.EnforceMutexStage(applyStage):50 - No mutex key specified for workload: d7e23ea6-d741-4314-9209-a33c91a2e945_15400100_0_check. Continuing...
2024-07-29 07:20:45 INFO i.a.w.l.p.s.m.Stage(apply):39 - APPLY Stage: LAUNCH — (workloadId = d7e23ea6-d741-4314-9209-a33c91a2e945_15400100_0_check) — (dataplaneId = aws-eu-paris-queue-1)
2024-07-29 07:20:54 INFO i.a.w.l.c.WorkloadApiClient(updateStatusToLaunched):60 - Attempting to update workload: d7e23ea6-d741-4314-9209-a33c91a2e945_15400100_0_check to LAUNCHED.
2024-07-29 07:20:54 INFO i.a.w.l.p.h.SuccessHandler(accept):60 - Pipeline completed for workload: d7e23ea6-d741-4314-9209-a33c91a2e945_15400100_0_check.
2024-07-29 07:20:59 platform > Failing job: 15400100, reason: Connection Check Failed 212cf3b7-ab21-4b41-b54b-501e4d688b67

Contribute

btkcodedev commented 3 months ago

@valentinfily Due to credential limitations, can you please take a pull of this URL branch and verify the working? Everything seems aligned with Trustpilot documentation