move-coop / parsons

A python library of connectors for the progressive community.
Other
254 stars 125 forks source link

[Bug] Multiple instances of Google connectors cannot be used simultaneously with different credentials #1039

Open austinweisgrau opened 2 months ago

austinweisgrau commented 2 months ago

When multiple instances of Google connectors are initialized, each successive connector overwrites the same GOOGLE_APPLICATION_CREDENTIALS environment variable, thereby overwriting the authentication of the prior connectors.

Detailed Description

parsons.google.utitities.setup_google_application_credentials() takes a dictionary of application credentials as an argument and stores them generically in the GOOGLE_APPLICATION_CREDENTIALS environment variable. This environment variable is then read by instances of GCP clients rather than passed directly to them. This makes it impossible to initialize multiple connectors with different credentials in the same code.

To Reproduce

from utilities.google import GoogleBigQuery

first_client = GoogleBigQuery(app_creds=first_app_creds, project="FIRST_PROJECT")
second_client = GoogleBigQuery(app_creds=second_app_creds, project="SECOND_PROJECT")

first_client.query("select session_user()")
>> Forbidden: 403 POST https://bigquery.googleapis.com/bigquery/v2/projects/FIRST_PROJECT/jobs?prettyPrint=false: Access Denied: Project FIRST_PROJECT: User does not have bigquery.jobs.create permission in project FIRST_PROJECT.

Your Environment

Additional Context

Add any other context about the problem here.

Priority

Please indicate whether fixing this bug is high, medium, or low priority for you. If the issue is time-sensitive for you, please let us know when you need it addressed by.