search

iconara / bigshift

A tool for moving tables from Redshift to BigQuery
BSD 3-Clause "New" or "Revised" License
65 stars 10 forks source link

TransferJob fails to start due to permissions #8

Closed iserko closed 8 years ago

iserko commented 8 years ago

I keep getting the following error when trying to execute the transfer step of bigshift

/usr/local/opt/rbenv/versions/2.3.0/lib/ruby/gems/2.3.0/gems/google-api-client-0.9.12/lib/google/apis/core/http_command.rb:211:in `check_status': failedPrecondition: Error creating the transfer job: Failed to obtain the location of the destination Google Cloud Storage (GCS) bucket due to insufficient permissions.  Please verify that the necessary permissions have been granted. (Google::Apis::ClientError)

The credentials I'm using are for a service account which has permissions to write to the GS bucket (tested writing with the same credentials) ... it also has Editor permissions on the Project, so I can't figure out what's wrong.

Any tips?

iconara commented 8 years ago

Sorry, I don't know enough about how the permissions in Google Cloud work. If you do resolve it please post back here for the next person who runs into the same problem.

iserko commented 8 years ago

@iconara did you use a Service Account when you ran this for yourself? I'm just curious if there was a special step you performed or was everything pretty much standard?

iconara commented 8 years ago

Yes, I used a service account. The account is a project editor, but I'm not sure how I can figure out what permissions it has. I'm not very familiar with Google Cloud's permissions model, unfortunately.

iserko commented 8 years ago

@iconara just made it work. I tried it via the UI just now ... and the UI added an additional permission to the bucket storage-transfer-<ID>@partnercontent.gserviceaccount.com ... didn't see anything about that anywhere. So now that I run bigshift it works perfectly.

iconara commented 8 years ago

Wow, that's really confusing.