search

triggermesh / tmctl

TriggerMesh Shaker CLI to create, manage and debug event-driven integration apps.
Apache License 2.0
16 stars 6 forks source link

Errors when creating a GoogleCloudStorage source with tmctl #288

Closed jmcx closed 1 year ago

jmcx commented 1 year ago

It seems like it is impossible today to create a GCS source with tmctl without hitting some errors.

How to reproduce:

  1. Create a service account called triggermesh in GCP IAM that has PubSub Admin and Storage Admin roles
  2. Create a key for the service account and save it as JSON
  3. Create a bucket called triggermesh-bucket

Then run 

tmctl create source googlecloudstorage --bucket triggermesh-bucket --pubsub.project <project-name> --serviceAccountKey $(cat serviceaccountkey.json)

Error:

2023/04/06 16:49:52 source initialization: Error obtaining notification configuration for bucket "triggermesh-bucket": adding notification configuration: googleapi: Error 403: The service account 'service-549199081752@gs-project-accounts.iam.gserviceaccount.com' does not have permission to publish messages to to the Cloud Pub/Sub topic '//pubsub.googleapis.com/projects/object-notifications/topics/local.triggermesh-googlecloudstoragesource~googlecloudstoragesources.sources.triggermesh.io', or that topic does not exist., forbidden

It looks like the bucket's service account service-549199081752... was not given permission to publish notifications to the pub/sub topic created by the triggermesh GCS source. Because this topic was created dynamically by the source, I cannot as a user have provisioned this permission beforehand. Based on a test @FranBarrera did, it sounds like the equivalent command in K8s by creating a GCS source CRD does work and is able to configure these permissions, so maybe there is an issue with the reconciliation logic?

If I manually add the missing permission stated above, then everything works. But it is expected that this should happen automatically.

tzununbekov commented 1 year ago

@jmcx can you check if the same configuration works in K8s for you? I don't see any difference in the CLI and tm/tm reconcilers.

jmcx commented 1 year ago

Indeed, same issue on K8s:

{ "lastTransitionTime": "2023-04-07T14:20:48Z", "message": "Access denied to Cloud Storage API: adding notification configuration: googleapi: Error 403: The service account 'service-549199081752@gs-project-accounts.iam.gserviceaccount.com' does not have permission to publish messages to to the Cloud Pub/Sub topic '//pubsub.googleapis.com/projects/object-notifications/topics/default.gcssource~googlecloudstoragesources.sources.triggermesh.io', or that topic does not exist., forbidden", "reason": "APIError", "status": "False", "type": "Subscribed" }

jmcx commented 1 year ago

shall we move this ticket to tm/tm ?

tzununbekov commented 1 year ago

shall we move this ticket to tm/tm ?

I think that it may end up as a documentation issue, but right now tm/tm would be a better fit

jmcx commented 1 year ago

Indeed: you had to give the Google Cloud Storage service account for the current project the Pub/Sub publisher role on the project to avoid the aforementioned error.

I'll update the docs: https://github.com/triggermesh/docs/issues/365