search

airbytehq / PyAirbyte

PyAirbyte brings the power of Airbyte to every Python developer.
https://docs.airbyte.com/pyairbyte
Other
176 stars 20 forks source link

source-google-analytics-data-api authentication - 'Client' was expected even only with service id #264

Open henryguyu opened 1 month ago

henryguyu commented 1 month ago

I have 'auth_type': 'Service' in setting. however it asked for 'Client' which is for oauth. below is the format of setting:

{
  "property_ids": [
    "..."
  ],
  "credentials": {
    "credentials_json": {
      "type": "service_account",
      "project_id": "...",
      "private_key_id": "...",
      "private_key": "...",
      "client_email": "...",
      "client_id": "...",
      "auth_uri": "https://accounts.google.com/o/oauth2/auth",
      "token_uri": "https://oauth2.googleapis.com/token",
      "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
      "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/development%40famous-cove-353314.iam.gserviceaccount.com"
    },
    "auth_type": "Service"
  },
  "date_ranges_start_date": "2024-01-01"
}

and error message:

File "/usr/local/lib/python3.9/site-packages/airbyte/sources/base.py", line 580, in _execute for line in self.executor.execute(args): File "/usr/local/lib/python3.9/site-packages/airbyte/_executor.py", line 413, in execute yield from stream File "/usr/local/lib/python3.9/contextlib.py", line 126, in __exit__ next(self.gen) File "/usr/local/lib/python3.9/site-packages/airbyte/_executor.py", line 142, in _stream_from_subprocess raise exc.AirbyteSubprocessFailedError(airbyte.exceptions.AirbyteSubprocessFailedError: AirbyteSubprocessFailedError: Subprocess failed. Run Args: ['/usr/local/dataflow/.venv-source-google-analytics-data-api/bin/source-google-analytics-data-api', 'discover', '--config', '/tmp/tmpz4ev3eh8.json'] Exit Code: 1The above exception was the direct cause of the following exception:Traceback (most recent call last): File "/usr/local/dataflow/airbyte_connector.py", line 256, in <module> read_data_to_cache(arguments, config, cache) File "/usr/local/dataflow/airbyte_connector.py", line 88, in read_data_to_cache source.select_streams(streams) File "/usr/local/lib/python3.9/site-packages/airbyte/sources/base.py", line 126, in select_streams available_streams = self.get_available_streams() File "/usr/local/lib/python3.9/site-packages/airbyte/sources/base.py", line 222, in get_available_streams return [s.name for s in self.discovered_catalog.streams] File "/usr/local/lib/python3.9/site-packages/airbyte/sources/base.py", line 320, in discovered_catalog self._discovered_catalog = self._discover() File "/usr/local/lib/python3.9/site-packages/airbyte/sources/base.py", line 183, in _discover for msg in self._execute(["discover", "--config", config_file]): File "/usr/local/lib/python3.9/site-packages/airbyte/sources/base.py", line 593, in _execute raise exc.AirbyteConnectorReadError(airbyte.exceptions.AirbyteConnectorReadError: AirbyteConnectorReadError: Error when reading from the connector.Log output: 'Client' was expected Traceback (most recent call last): File "/usr/local/dataflow/.venv-source-google-analytics-data-api/bin/source-google-analytics-data-api", line 8, in <module> sys.exit(run()) File "/usr/local/dataflow/.venv-source-google-analytics-data-api/lib/python3.9/site-packages/source_google_analytics_data_api/run.py", line 18, in run launch(source, sys.argv[1:]) File "/usr/local/dataflow/.venv-source-google-analytics-data-api/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 235, in launch for message in source_entrypoint.run(parsed_args): File "/usr/local/dataflow/.venv-source-google-analytics-data-api/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 117, in run yield from map(AirbyteEntrypoint.airbyte_message_to_string, self.discover(source_spec, config)) File "/usr/local/dataflow/.venv-source-google-analytics-data-api/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 151, in discover self.validate_connection(source_spec, config) File "/usr/local/dataflow/.venv-source-google-analytics-data-api/lib/python3.9/site-packages/airbyte_cdk/entrypoint.py", line 192, in validate_connection check_config_against_spec_or_exit(connector_config, source_spec) File "/usr/local/dataflow/.venv-source-google-analytics-data-api/lib/python3.9/site-packages/airbyte_cdk/sources/utils/schema_helpers.py", line 175, in check_config_against_spec_or_exit raise AirbyteTracedException( airbyte_cdk.utils.traced_exception.AirbyteTracedException: 'Client' was expected Config validation error: 'Client' was expected

aaronsteers commented 1 month ago

@henryguyu - thanks for reporting this. I'm not familiar with this connector and it is hard to provide immediate direction. Have you been able to locate the spec output for this connector and compare with the inputs you are provided? Can you print this with the following for comparison, and then paste that text back here to the issue if it doesn't sufficiently direct you to a resolution?

source = get_source(...)
source.print_config_spec()

Docs here: https://airbytehq.github.io/PyAirbyte/airbyte/sources.html#Source.print_config_spec

henryguyu commented 1 month ago

@aaronsteers thanks for your response. below is the spec I got with your suggestion. I consider my config meets the specification. can you help me to ensure it? my config is at the beginning of my first post.

<airbyte.sources.base.Source object at 0x7f2de55f3670> $schema: https://json-schema.org/draft-07/schema# additionalProperties: true properties: convert_conversions_event: default: false description: Enables conversion of conversions:* event metrics from intege to floats. This is beneficial for preventing data rounding when the API re float values for any conversions:* fields. order: 7 title: Convert conversions:* Metrics to Float type: boolean credentials: description: Credentials for the service oneOf:

henryguyu commented 1 month ago

@aaronsteers is there any progress? thanks so much!