search

MartijnSch / amplitude-bigquery

Export your events from Amplitude to Google BigQuery/Google Cloud Storage
MIT License
31 stars 18 forks source link

Integration issue with Amplitude and GCS @MartijnSch #1

Open akshafmulla opened 5 years ago

akshafmulla commented 5 years ago

Hi MartijnSch,

I am a beginner in this field. I have a requirement where we need to export the from Amplitude to GCS Bigquery.

I have used your code to do the export and import part.

I did all the required configurations, but it is throwing me an error while i am running this script.

The Error is as below:

Traceback (most recent call last):

File "", line 179, in bigquery_client = bigquery.Client(project=PROJECT_ID)

File "C:\ProgramData\Anaconda3\lib\site-packages\google\cloud\bigquery\client.py", line 165, in init project=project, credentials=credentials, _http=_http

File "C:\ProgramData\Anaconda3\lib\site-packages\google\cloud\client.py", line 228, in init Client.init(self, credentials=credentials, _http=_http)

File "C:\ProgramData\Anaconda3\lib\site-packages\google\cloud\client.py", line 133, in init credentials, _ = google.auth.default()

File "C:\ProgramData\Anaconda3\lib\site-packages\google\auth_default.py", line 317, in default raise exceptions.DefaultCredentialsError(_HELP_MESSAGE)

DefaultCredentialsError: Could not automatically determine credentials. Please set GOOGLE_APPLICATION_CREDENTIALS or explicitly create credentials and re-run the application. For more information, please see https://cloud.google.com/docs/authentication/getting-started

Also, I am unable to find the Account_Id as required in the program. What type of ID it is?

Thanks a ton for all your support.

Regards, Akshaf Mulla

MartijnSch commented 5 years ago

@akshafmulla It seems that the Project ID is for sure missing and that you haven't installed either Python or the packages that might be needed to authenticate with Google Cloud Platform. All the other credentials that you have to fill in are directly coming from there and are currently causing your errors.

akshafmulla commented 5 years ago

Hey Martin,

I have provided the required keys and project id. I cant find any login integration into GCS except for PROJECT_ID and CLOUD_STORAGE_BUCKET. How is our code authenticating into GCS?

MartijnSch commented 5 years ago

Your Google Cloud project credentials will get you into Google Cloud Storage too. The Project ID is the one that is related to Google Cloud Platform and the Storage Bucket is the name of the bucket you should create there.

nickerhov89 commented 5 years ago

Hey Martin,

I have provided the required keys and project id. I cant find any login integration into GCS except for PROJECT_ID and CLOUD_STORAGE_BUCKET. How is our code authenticating into GCS?

Add this: import os from google.cloud.bigquery.client import Client

os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = 'YOUR_API_KEY.json' bq_client = Client()

Before this: bigquery_client = bigquery.Client(project=PROJECT_ID)