[source-sharepoint] connector failes check when used with SHARED_ITEMS

akozichev commented 2 months ago

Connector Name

source-microsoft-sharepoint

Connector Version

0.2.2

What step the error happened?

Configuring a new connector

Relevant information

I am using Sharepoint connector to sync files from Sharepoint into Azure Storage.
I am using Service Key Authentication. I am not able to use Search Scope set to "SHARED_ITEMS" or "ALL" as the connector fails to pass the check mechanism. Credentials provided have full file.read access in graph API.

Message "/me request is only valid with delegated authentication flow". tells that /me endpoint won't work with this type authentication.

Relevant log output

{"type": "LOG", "log": {"level": "ERROR", "message": "Check failed"}}
{"type": "CONNECTION_STATUS", "connectionStatus": {"status": "FAILED", "message": "['Traceback (most recent call last):\\n  File \"/usr/local/lib/python3.9/site-packages/office365/runtime/client_request.py\", line 38, in execute_query\\n    response.raise_for_status()\\n  File \"/usr/local/lib/python3.9/site-packages/requests/models.py\", line 1021, in raise_for_status\\n    raise HTTPError(http_error_msg, response=self)\\nrequests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://graph.microsoft.com/v1.0/me/drive/sharedWithMe\\n\\nDuring handling of the above exception, another exception occurred:\\n\\nTraceback (most recent call last):\\n  File \"/airbyte/integration_code/source_microsoft_sharepoint/utils.py\", line 58, in execute_query_with_retry\\n    return obj.execute_query()\\n  File \"/usr/local/lib/python3.9/site-packages/office365/runtime/client_object.py\", line 52, in execute_query\\n    self.context.execute_query()\\n  File \"/usr/local/lib/python3.9/site-packages/office365/runtime/client_runtime_context.py\", line 183, in execute_query\\n    self.pending_request().execute_query(qry)\\n  File \"/usr/local/lib/python3.9/site-packages/office365/runtime/client_request.py\", line 42, in execute_query\\n    raise ClientRequestException(*e.args, response=e.response)\\noffice365.runtime.client_request_exception.ClientRequestException: (\\'BadRequest\\', \\'/me request is only valid with delegated authentication flow.\\', \\'400 Client Error: Bad Request for url: https://graph.microsoft.com/v1.0/me/drive/sharedWithMe\\')\\n\\nDuring handling of the above exception, another exception occurred:\\n\\nTraceback (most recent call last):\\n  File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/availability_strategy/default_file_based_availability_strategy.py\", line 81, in _check_list_files\\n    file = next(iter(stream.get_files()))\\n  File \"/airbyte/integration_code/source_microsoft_sharepoint/stream_reader.py\", line 243, in get_matching_files\\n    [\\n  File \"/airbyte/integration_code/source_microsoft_sharepoint/stream_reader.py\", line 243, in <listcomp>\\n    [\\n  File \"/airbyte/integration_code/source_microsoft_sharepoint/stream_reader.py\", line 233, in get_all_files\\n    yield from self._get_shared_files_from_all_drives(parsed_drives)\\n  File \"/airbyte/integration_code/source_microsoft_sharepoint/stream_reader.py\", line 216, in _get_shared_files_from_all_drives\\n    shared_drive_items = execute_query_with_retry(self.one_drive_client.me.drive.shared_with_me())\\n  File \"/airbyte/integration_code/source_microsoft_sharepoint/utils.py\", line 82, in execute_query_with_retry\\n    raise AirbyteTracedException.from_exception(ex, message=\"Caught unexpected exception\")\\nairbyte_cdk.utils.traced_exception.AirbyteTracedException: (\\'BadRequest\\', \\'/me request is only valid with delegated authentication flow.\\', \\'400 Client Error: Bad Request for url: https://graph.microsoft.com/v1.0/me/drive/sharedWithMe\\')\\n\\nThe above exception was the direct cause of the following exception:\\n\\nTraceback (most recent call last):\\n  File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/availability_strategy/default_file_based_availability_strategy.py\", line 62, in check_availability_and_parsability\\n    file = self._check_list_files(stream)\\n  File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/sources/file_based/availability_strategy/default_file_based_availability_strategy.py\", line 87, in _check_list_files\\n    raise CheckAvailabilityError(FileBasedSourceError.ERROR_LISTING_FILES, stream=stream.name) from exc\\nairbyte_cdk.sources.file_based.exceptions.CheckAvailabilityError: Error listing files. Please check the credentials provided in the config and verify that they provide permission to list files. Contact Support if you need assistance.\\nstream=old\\n']"}}