search

airbytehq / PyAirbyte

PyAirbyte brings the power of Airbyte to every Python developer.
https://docs.airbyte.com/pyairbyte
Other
178 stars 20 forks source link

Feat: Adapt BigQuery classes to be able to use default credential #157

Closed 4sushi closed 3 months ago

4sushi commented 3 months ago

Goal

Adapt BigQueryCache to be able to connect without a service account, by using default credential. Resolved: https://github.com/airbytehq/PyAirbyte/discussions/129

Example:

import airbyte
from airbyte.caches.bigquery import BigQueryCache
source: airbyte.Source = airbyte.get_source("source-faker")
source.set_config(config={"count": 10_000,  "seed": 123,},)
source.check()

bq_cache = BigQueryCache(project_name='xxx', dataset_name='test')

source.select_all_streams()
read_result: airbyte.ReadResult = source.read(cache=bq_cache)

Note:

I was not able to run the tests locally. I just tested the code above. Feel free to adapt the code or put some comments if I miss something.

aaronsteers commented 3 months ago

/test-pr

aaronsteers commented 3 months ago

/fix-pr

aaronsteers commented 3 months ago

/fix-pr

aaronsteers commented 3 months ago

@4sushi - Thanks for submitting this. I will be reviewing today and tomorrow.

Note I've auto-applied lint+format fixes, and I've pulled the latest from main branch. You may need to pull these updates before committing/pushing any further updates.

aaronsteers commented 3 months ago

/fix-pr

PR auto-fix job started... Check job output.

✅ Changes applied successfully.

aaronsteers commented 3 months ago

/test-pr

PR test job started... Check job output.

❌ Tests failed.

aaronsteers commented 3 months ago

/fix-pr

PR auto-fix job started... Check job output.

🟦 Job completed successfully (no changes).

aaronsteers commented 3 months ago

/test-pr

PR test job started... Check job output.

❌ Tests failed.

aaronsteers commented 3 months ago

/test-pr

PR test job started... Check job output.

❌ Tests failed.

aaronsteers commented 3 months ago

/fix-pr

PR auto-fix job started... Check job output.

✅ Changes applied successfully.

aaronsteers commented 3 months ago

/test-pr

PR test job started... Check job output.

✅ Tests passed.