This removes some of the noise that would obscure the actual error from the connector. Tested with google sheets and the output of this connector becomes this:
$ flowctl raw discover ghcr.io/estuary/source-google-sheets:21e4a87 acmeCo
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
{"ts":"2023-10-27T12:29:06.847252138+00:00","level":"info","message":"connector-init started","fields":{"log_level":"info","module":"connector_init","port":49092}}
{"ts":"2023-10-27T12:29:09.882460709+00:00","level":"info","message":"Running discovery on sheet blah"}
{"ts":"2023-10-27T12:29:10.373776042+00:00","level":"error","message":"Giving up get(...) after 1 tries (googleapiclient.errors.HttpError: <HttpError 404 when requesting https://sheets.googleapis.com/v4/spreadsheets/blah?includeGridData=false&alt=json returned \"Requested entity was not found.\". Details: \"Requested entity was not found.\">)"}
{"ts":"2023-10-27T12:29:10.387711959+00:00","level":"error","message":"Could not run discovery: Requested spreadsheet was not found.\nTraceback (most recent call last):\n File \"/airbyte/integration_code/google_sheets_source/google_sheets_source.py\", line 107, in discover\n spreadsheet_metadata = Spreadsheet.parse_obj(client.get(spreadsheetId=spreadsheet_id, includeGridData=False))\n File \"/usr/local/lib/python3.9/site-packages/backoff/_sync.py\", line 105, in retry\n ret = target(*args, **kwargs)\n File \"/airbyte/integration_code/google_sheets_source/client.py\", line 27, in get\n return self.client.get(**kwargs).execute()\n File \"/usr/local/lib/python3.9/site-packages/googleapiclient/_helpers.py\", line 130, in positional_wrapper\n return wrapped(*args, **kwargs)\n File \"/usr/local/lib/python3.9/site-packages/googleapiclient/http.py\", line 938, in execute\n raise HttpError(resp, content, uri=self.uri)\ngoogleapiclient.errors.HttpError: <HttpError 404 when requesting https://sheets.googleapis.com/v4/spreadsheets/blah?includeGridData=false&alt=json returned \"Requested entity was not found.\". Details: \"Requested entity was not found.\">\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/airbyte/integration_code/main.py\", line 13, in <module>\n launch(source, sys.argv[1:])\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 200, in launch\n for message insource_entrypoint.run(parsed_args):\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 109, in run\n yield from map(AirbyteEntrypoint.airbyte_message_to_string, self.discover(source_spec, config))\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 144, in discover\n catalog = self.source.discover(self.logger, config)\n File \"/airbyte/integration_code/google_sheets_source/google_sheets_source.py\", line 126, in discover\n raise Exception(f\"Could not run discovery: {reason}\")\nException: Could not run discovery: Requested spreadsheet was not found."}
{"ts":"2023-10-27T12:29:10.461988043+00:00","level":"error","message":"connector failed","fields":{"module":"connector_init::rpc","status":"exit status: 1"}}
Error: connector discover
Caused by:
status: Internal, message: "Could not run discovery: Requested spreadsheet was not found.\nTraceback (most recent call last):\n File \"/airbyte/integration_code/google_sheets_source/google_sheets_source.py\", line 107, in discover\n spreadsheet_metadata = Spreadsheet.parse_obj(client.get(spreadsheetId=spreadsheet_id, includeGridData=False))\n File \"/usr/local/lib/python3.9/site-packages/backoff/_sync.py\", line 105, in retry\n ret = target(args, kwargs)\n File \"/airbyte/integration_code/google_sheets_source/client.py\", line 27, in get\n return self.client.get(kwargs).execute()\n File \"/usr/local/lib/python3.9/site-packages/googleapiclient/_helpers.py\", line 130, in positional_wrapper\n return wrapped(args, **kwargs)\n File \"/usr/local/lib/python3.9/site-packages/googleapiclient/http.py\", line 938, in execute\n raise HttpError(resp, content, uri=self.uri)\ngoogleapiclient.errors.HttpError: <HttpError 404 when requesting https://sheets.googleapis.com/v4/spreadsheets/blah?includeGridData=false&alt=json returned \"Requested entitywas not found.\". Details: \"Requested entity was not found.\">\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/airbyte/integration_code/main.py\", line 13, in \n launch(source, sys.argv[1:])\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 200, in launch\n for message in source_entrypoint.run(parsed_args):\nFile \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 109, in run\n yield from map(AirbyteEntrypoint.airbyte_message_to_string, self.discover(source_spec, config))\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 144, in discover\n catalog = self.source.discover(self.logger, config)\n File \"/airbyte/integration_code/google_sheets_source/google_sheets_source.py\", line 126, in discover\n raise Exception(f\"Could not run discovery: {reason}\")\nException: Could not run discovery: Requested spreadsheet was not found.", details: [], metadata: MetadataMap { headers: {"last-log-bin": "GgwIltruqQYQ14fwuAEgASrhD0NvdWxkIG5vdCBydW4gZGlzY292ZXJ5OiBSZXF1ZXN0ZWQgc3ByZWFkc2hlZXQgd2FzIG5vdCBmb3VuZC4KVHJhY2ViYWNrIChtb3N0IHJlY2VudCBjYWxsIGxhc3QpOgogIEZpbGUgIi9haXJieXRlL2ludGVncmF0aW9uX2NvZGUvZ29vZ2xlX3NoZWV0c19zb3VyY2UvZ29vZ2xlX3NoZWV0c19zb3VyY2UucHkiLCBsaW5lIDEwNywgaW4gZGlzY292ZXIKICAgIHNwcmVhZHNoZWV0X21ldGFkYXRhID0gU3ByZWFkc2hlZXQucGFyc2Vfb2JqKGNsaWVudC5nZXQoc3ByZWFkc2hlZXRJZD1zcHJlYWRzaGVldF9pZCwgaW5jbHVkZUdyaWREYXRhPUZhbHNlKSkKICBGaWxlICIvdXNyL2xvY2FsL2xpYi9weXRob24zLjkvc2l0ZS1wYWNrYWdlcy9iYWNrb2ZmL19zeW5jLnB5IiwgbGluZSAxMDUsIGluIHJldHJ5CiAgICByZXQgPSB0YXJnZXQoKmFyZ3MsICoqa3dhcmdzKQogIEZpbGUgIi9haXJieXRlL2ludGVncmF0aW9uX2NvZGUvZ29vZ2xlX3NoZWV0c19zb3VyY2UvY2xpZW50LnB5IiwgbGluZSAyNywgaW4gZ2V0CiAgICByZXR1cm4gc2VsZi5jbGllbnQuZ2V0KCoqa3dhcmdzKS5leGVjdXRlKCkKICBGaWxlICIvdXNyL2xvY2FsL2xpYi9weXRob24zLjkvc2l0ZS1wYWNrYWdlcy9nb29nbGVhcGljbGllbnQvX2hlbHBlcnMucHkiLCBsaW5lIDEzMCwgaW4gcG9zaXRpb25hbF93cmFwcGVyCiAgICByZXR1cm4gd3JhcHBlZCgqYXJncywgKiprd2FyZ3MpCiAgRmlsZSAiL3Vzci9sb2NhbC9saWIvcHl0aG9uMy45L3NpdGUtcGFja2FnZXMvZ29vZ2xlYXBpY2xpZW50L2h0dHAucHkiLCBsaW5lIDkzOCwgaW4gZXhlY3V0ZQogICAgcmFpc2UgSHR0cEVycm9yKHJlc3AsIGNvbnRlbnQsIHVyaT1zZWxmLnVyaSkKZ29vZ2xlYXBpY2xpZW50LmVycm9ycy5IdHRwRXJyb3I6IDxIdHRwRXJyb3IgNDA0IHdoZW4gcmVxdWVzdGluZyBodHRwczovL3NoZWV0cy5nb29nbGVhcGlzLmNvbS92NC9zcHJlYWRzaGVldHMvYmxhaD9pbmNsdWRlR3JpZERhdGE9ZmFsc2UmYWx0PWpzb24gcmV0dXJuZWQgIlJlcXVlc3RlZCBlbnRpdHkgd2FzIG5vdCBmb3VuZC4iLiBEZXRhaWxzOiAiUmVxdWVzdGVkIGVudGl0eSB3YXMgbm90IGZvdW5kLiI+CgpEdXJpbmcgaGFuZGxpbmcgb2YgdGhlIGFib3ZlIGV4Y2VwdGlvbiwgYW5vdGhlciBleGNlcHRpb24gb2NjdXJyZWQ6CgpUcmFjZWJhY2sgKG1vc3QgcmVjZW50IGNhbGwgbGFzdCk6CiAgRmlsZSAiL2FpcmJ5dGUvaW50ZWdyYXRpb25fY29kZS9tYWluLnB5IiwgbGluZSAxMywgaW4gPG1vZHVsZT4KICAgIGxhdW5jaChzb3VyY2UsIHN5cy5hcmd2WzE6XSkKICBGaWxlICIvdXNyL2xvY2FsL2xpYi9weXRob24zLjkvc2l0ZS1wYWNrYWdlcy9haXJieXRlX2Nkay9lbnRyeXBvaW50LnB5IiwgbGluZSAyMDAsIGluIGxhdW5jaAogICAgZm9yIG1lc3NhZ2UgaW4gc291cmNlX2VudHJ5cG9pbnQucnVuKHBhcnNlZF9hcmdzKToKICBGaWxlICIvdXNyL2xvY2FsL2xpYi9weXRob24zLjkvc2l0ZS1wYWNrYWdlcy9haXJieXRlX2Nkay9lbnRyeXBvaW50LnB5IiwgbGluZSAxMDksIGluIHJ1bgogICAgeWllbGQgZnJvbSBtYXAoQWlyYnl0ZUVudHJ5cG9pbnQuYWlyYnl0ZV9tZXNzYWdlX3RvX3N0cmluZywgc2VsZi5kaXNjb3Zlcihzb3VyY2Vfc3BlYywgY29uZmlnKSkKICBGaWxlICIvdXNyL2xvY2FsL2xpYi9weXRob24zLjkvc2l0ZS1wYWNrYWdlcy9haXJieXRlX2Nkay9lbnRyeXBvaW50LnB5IiwgbGluZSAxNDQsIGluIGRpc2NvdmVyCiAgICBjYXRhbG9nID0gc2VsZi5zb3VyY2UuZGlzY292ZXIoc2VsZi5sb2dnZXIsIGNvbmZpZykKICBGaWxlICIvYWlyYnl0ZS9pbnRlZ3JhdGlvbl9jb2RlL2dvb2dsZV9zaGVldHNfc291cmNlL2dvb2dsZV9zaGVldHNfc291cmNlLnB5IiwgbGluZSAxMjYsIGluIGRpc2NvdmVyCiAgICByYWlzZSBFeGNlcHRpb24oZiJDb3VsZCBub3QgcnVuIGRpc2NvdmVyeToge3JlYXNvbn0iKQpFeGNlcHRpb246IENvdWxkIG5vdCBydW4gZGlzY292ZXJ5OiBSZXF1ZXN0ZWQgc3ByZWFkc2hlZXQgd2FzIG5vdCBmb3VuZC4"} }
Caused by: status: Internal, message: "Could not run discovery: Requested spreadsheet was not found.\nTraceback (most recent call last):\n File \"/airbyte/integration_code/google_sheets_source/google_sheets_source.py\", line 107, in discover\n spreadsheet_metadata = Spreadsheet.parse_obj(client.get(spreadsheetId=spreadsheet_id, includeGridData=False))\n File \"/usr/local/lib/python3.9/site-packages/backoff/_sync.py\", line 105, in retry\n ret = target(args, kwargs)\n File \"/airbyte/integration_code/google_sheets_source/client.py\", line 27, in get\n return self.client.get(kwargs).execute()\n File \"/usr/local/lib/python3.9/site-packages/googleapiclient/_helpers.py\", line 130, in positional_wrapper\n return wrapped(args, **kwargs)\n File \"/usr/local/lib/python3.9/site-packages/googleapiclient/http.py\", line 938, in execute\n raise HttpError(resp, content, uri=self.uri)\ngoogleapiclient.errors.HttpError: <HttpError 404 when requesting https://sheets.googleapis.com/v4/spreadsheets/blah?includeGridData=false&alt=json returned \"Requested entitywas not found.\". Details: \"Requested entity was not found.\">\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/airbyte/integration_code/main.py\", line 13, in\n launch(source, sys.argv[1:])\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 200, in launch\n for message in source_entrypoint.run(parsed_args):\nFile \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 109, in run\n yield from map(AirbyteEntrypoint.airbyte_message_to_string, self.discover(source_spec, config))\n File \"/usr/local/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py\", line 144, in discover\n catalog = self.source.discover(self.logger, config)\n File \"/airbyte/integration_code/google_sheets_source/google_sheets_source.py\", line 126, in discover\n raise Exception(f\"Could not run discovery: {reason}\")\nException: Could not run discovery: Requested spreadsheet was not found.", details: [], metadata: MetadataMap { headers: {"last-log-bin": "GgwIltruqQYQ14fwuAEgASrhD0NvdWxkIG5vdCBydW4gZGlzY292ZXJ5OiBSZXF1ZXN0ZWQgc3ByZWFkc2hlZXQgd2FzIG5vdCBmb3VuZC4KVHJhY2ViYWNrIChtb3N0IHJlY2VudCBjYWxsIGxhc3QpOgogIEZpbGUgIi9haXJieXRlL2ludGVncmF0aW9uX2NvZGUvZ29vZ2xlX3NoZWV0c19zb3VyY2UvZ29vZ2xlX3NoZWV0c19zb3VyY2UucHkiLCBsaW5lIDEwNywgaW4gZGlzY292ZXIKICAgIHNwcmVhZHNoZWV0X21ldGFkYXRhID0gU3ByZWFkc2hlZXQucGFyc2Vfb2JqKGNsaWVudC5nZXQoc3ByZWFkc2hlZXRJZD1zcHJlYWRzaGVldF9pZCwgaW5jbHVkZUdyaWREYXRhPUZhbHNlKSkKICBGaWxlICIvdXNyL2xvY2FsL2xpYi9weXRob24zLjkvc2l0ZS1wYWNrYWdlcy9iYWNrb2ZmL19zeW5jLnB5IiwgbGluZSAxMDUsIGluIHJldHJ5CiAgICByZXQgPSB0YXJnZXQoKmFyZ3MsICoqa3dhcmdzKQogIEZpbGUgIi9haXJieXRlL2ludGVncmF0aW9uX2NvZGUvZ29vZ2xlX3NoZWV0c19zb3VyY2UvY2xpZW50LnB5IiwgbGluZSAyNywgaW4gZ2V0CiAgICByZXR1cm4gc2VsZi5jbGllbnQuZ2V0KCoqa3dhcmdzKS5leGVjdXRlKCkKICBGaWxlICIvdXNyL2xvY2FsL2xpYi9weXRob24zLjkvc2l0ZS1wYWNrYWdlcy9nb29nbGVhcGljbGllbnQvX2hlbHBlcnMucHkiLCBsaW5lIDEzMCwgaW4gcG9zaXRpb25hbF93cmFwcGVyCiAgICByZXR1cm4gd3JhcHBlZCgqYXJncywgKiprd2FyZ3MpCiAgRmlsZSAiL3Vzci9sb2NhbC9saWIvcHl0aG9uMy45L3NpdGUtcGFja2FnZXMvZ29vZ2xlYXBpY2xpZW50L2h0dHAucHkiLCBsaW5lIDkzOCwgaW4gZXhlY3V0ZQogICAgcmFpc2UgSHR0cEVycm9yKHJlc3AsIGNvbnRlbnQsIHVyaT1zZWxmLnVyaSkKZ29vZ2xlYXBpY2xpZW50LmVycm9ycy5IdHRwRXJyb3I6IDxIdHRwRXJyb3IgNDA0IHdoZW4gcmVxdWVzdGluZyBodHRwczovL3NoZWV0cy5nb29nbGVhcGlzLmNvbS92NC9zcHJlYWRzaGVldHMvYmxhaD9pbmNsdWRlR3JpZERhdGE9ZmFsc2UmYWx0PWpzb24gcmV0dXJuZWQgIlJlcXVlc3RlZCBlbnRpdHkgd2FzIG5vdCBmb3VuZC4iLiBEZXRhaWxzOiAiUmVxdWVzdGVkIGVudGl0eSB3YXMgbm90IGZvdW5kLiI+CgpEdXJpbmcgaGFuZGxpbmcgb2YgdGhlIGFib3ZlIGV4Y2VwdGlvbiwgYW5vdGhlciBleGNlcHRpb24gb2NjdXJyZWQ6CgpUcmFjZWJhY2sgKG1vc3QgcmVjZW50IGNhbGwgbGFzdCk6CiAgRmlsZSAiL2FpcmJ5dGUvaW50ZWdyYXRpb25fY29kZS9tYWluLnB5IiwgbGluZSAxMywgaW4gPG1vZHVsZT4KICAgIGxhdW5jaChzb3VyY2UsIHN5cy5hcmd2WzE6XSkKICBGaWxlICIvdXNyL2xvY2FsL2xpYi9weXRob24zLjkvc2l0ZS1wYWNrYWdlcy9haXJieXRlX2Nkay9lbnRyeXBvaW50LnB5IiwgbGluZSAyMDAsIGluIGxhdW5jaAogICAgZm9yIG1lc3NhZ2UgaW4gc291cmNlX2VudHJ5cG9pbnQucnVuKHBhcnNlZF9hcmdzKToKICBGaWxlICIvdXNyL2xvY2FsL2xpYi9weXRob24zLjkvc2l0ZS1wYWNrYWdlcy9haXJieXRlX2Nkay9lbnRyeXBvaW50LnB5IiwgbGluZSAxMDksIGluIHJ1bgogICAgeWllbGQgZnJvbSBtYXAoQWlyYnl0ZUVudHJ5cG9pbnQuYWlyYnl0ZV9tZXNzYWdlX3RvX3N0cmluZywgc2VsZi5kaXNjb3Zlcihzb3VyY2Vfc3BlYywgY29uZmlnKSkKICBGaWxlICIvdXNyL2xvY2FsL2xpYi9weXRob24zLjkvc2l0ZS1wYWNrYWdlcy9haXJieXRlX2Nkay9lbnRyeXBvaW50LnB5IiwgbGluZSAxNDQsIGluIGRpc2NvdmVyCiAgICBjYXRhbG9nID0gc2VsZi5zb3VyY2UuZGlzY292ZXIoc2VsZi5sb2dnZXIsIGNvbmZpZykKICBGaWxlICIvYWlyYnl0ZS9pbnRlZ3JhdGlvbl9jb2RlL2dvb2dsZV9zaGVldHNfc291cmNlL2dvb2dsZV9zaGVldHNfc291cmNlLnB5IiwgbGluZSAxMjYsIGluIGRpc2NvdmVyCiAgICByYWlzZSBFeGNlcHRpb24oZiJDb3VsZCBub3QgcnVuIGRpc2NvdmVyeToge3JlYXNvbn0iKQpFeGNlcHRpb246IENvdWxkIG5vdCBydW4gZGlzY292ZXJ5OiBSZXF1ZXN0ZWQgc3ByZWFkc2hlZXQgd2FzIG5vdCBmb3VuZC4"} }