Can't deploy Eventarc-triggered Cloud Function v2 with disabled default compute service account - How to specify service account used for Eventarc trigger?

jketcham commented 1 year ago

Version info

node: v18.14.2

firebase-functions: 4.3.1

firebase-tools: 12.4.5

firebase-admin: 11.8.0

Test case

See below.

Steps to reproduce

Not exactly sure all the factors at play here, but I'm creating a v2 Cloud Function with node 18, using the firebase deploy command with firestore eventarc triggers.

The default compute service account in this project has been disabled, as recommended for security purposes. I have specified the serviceAccount option, both at the individual function level and as a 'global option' to use another service account in the same project, but for some reason I still get an error complaining about the default service account not being valid. This is apparently due to the default compute account being used for the eventarc triggers.

I've granted my other service account I'm trying to use the 'Cloud Datastore User', 'Cloud Run Invoker' and 'Eventarc Event Receiver' roles.

Expected behavior

Deploy works without complaining about the default compute service account which is not referenced anywhere in my setup.

Actual behavior

When running firebase deploy --only functions in a fresh repo setup with firebase init functions, I get this error:

⚠  functions: HTTP Error: 400, Validation failed for trigger projects/prj-XXX/locations/nam5/triggers/mytrigger-123456: The request was invalid: invalid service account YYY-compute@developer.gserviceaccount.com provided

Despite trying to set my other service account via:

import { setGlobalOptions } from "firebase-functions/v2";

setGlobalOptions({
  serviceAccount: "sa-firestore-events@prj-XXX.iam.gserviceaccount.com",
});

and

export const myFunction = onDocumentCreated({
  document: "/documents/{docId}",
  serviceAccount: "sa-firestore-events@prj-XXX.iam.gserviceaccount.com",
}, (event) => { ... });

For reference, this is what the returned endpoint configuration looks like for the function I'm trying to deploy:

{
  availableMemoryMb: ResetValue {},
  timeoutSeconds: ResetValue {},
  minInstances: ResetValue {},
  maxInstances: ResetValue {},
  ingressSettings: ResetValue {},
  concurrency: ResetValue {},
  serviceAccountEmail: 'sa-firestore-events@prj-XXX.iam.gserviceaccount.com',
  vpc: ResetValue {},
  platform: 'gcfv2',
  labels: { service: 'firestore-events' },
  eventTrigger: {
    eventType: 'google.cloud.firestore.document.v1.created',
    eventFilters: { database: '(default)', namespace: '(default)' },
    eventFilterPathPatterns: { document: 'documents/{docId}' },
    retry: false
  }
}

Were you able to successfully deploy your functions?

I temporarily re-enabled the default compute service account, and after doing so with no other changes to my code, the deploy command worked, and the functions were created with the service account I specified.

I can see that the Eventarc triggers were created with the default compute service account.

How can I specify a different service account to use for the Eventarc triggers?

google-oss-bot commented 1 year ago

I found a few problems with this issue:

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.
This issue does not seem to follow the issue template. Make sure you provide all the required information.

aablsk commented 1 year ago

We are facing the same issue while evaluating how we can migrate from EventArc + CFv2 to Firebase Functions as we'd like to take advantage of the local emulator suite and some other niceties in Firebase compared to the under-the-hood GCP products.

We'd also like to be able to specify at least the service account for the EventArc trigger, but ideally we would love to have additional control such as retry behavior and dead letter queueing (which we'll otherwise manage through imported terraform resources)

Thank you for your time and consideration! 🙇

mvarchdev commented 1 year ago

Similiar issue here:

⚠  functions: HTTP Error: 400, Validation failed for trigger projects/glassfinder-develop-uhsnm/locations/europe-west3/triggers/onglassesupdateindexalgolia-342010: The request was invalid: invalid argument: event type google.cloud.firestore.document.v1.written not supported: attribute topic not found within event type
⚠  functions:  failed to update function projects/glassfinder-develop-uhsnm/locations/europe-west3/functions/onGlassesUpdateIndexAlgolia

Error: Failed to update function projects/glassfinder-develop-uhsnm/locations/europe-west3/functions/onGlassesUpdateIndexAlgolia
Error: Process completed with exit code 1.

russellwheatley commented 1 year ago

Hello 👋 @jketcham, I took a look at this issue. As you mentioned, I disabled the default compute service account & even the App Engine default service account as a sanity check.

I created a custom service account with the 'Cloud Datastore User', 'Cloud Run Invoker' and 'Eventarc Event Receiver' roles enabled.

I was able to deploy using both methods you mentioned:


// Method 1
setGlobalOptions({
  serviceAccount: "firestore-event-arc-test@***.iam.gserviceaccount.com",
});

 exports.function_with_custom_service = onDocumentCreated({
  document: "/documents/{docId}",
// Method 3
  // serviceAccount: "firestore-event-arc-test@***.iam.gserviceaccount.com",
}, (event) => { 
  logger.info(`Event received`);
 });

I am using the master branch of firebase-tools for clarity. Perhaps you should try the latest version (12.7.0) and let me know if you're still experiencing this issue? Thanks.

Having a look through the PRs, I noticed this one that has already been released might be what resolved your issue: https://github.com/firebase/firebase-tools/pull/6389

sconix commented 1 year ago

We have the same issue, unable to deploy v2 firestore trigger functions with custom service account. No matter if the service account is passed in the globaloptions or function options the deploy always fails with complaint about the default compute service account being missing (which it is). Using 12.7.0 version of firebase-tools and have granted exactly the same roles for the custom service account as stated above message from @russellwheatley.

Same thing with onSchedule functions, so far only onCall has worked, but I guess it does not require compute engine service account. By looking at the firebase-tools code the compute service account is hard coded and even by hacking the code the deploy still fails in function creation phase with the compute engine account not found error, so I guess there is no way to use custom service account currently :(

russellwheatley commented 11 months ago

@sconix - could you provide a reproduction, please. I was not able to reproduce. Thanks!

sconix commented 11 months ago

@sconix - could you provide a reproduction, please. I was not able to reproduce. Thanks!

@russellwheatley sorry for not beeing fully clear, so for us the deploy works if the SA is disabled, but two of our projects its completely missing and we have no means to create it (we are waiting for Google to resolve this issue i.e. fix our projects, but thats a secondary matter, imho Firebase should work with custom service accounts since thats Googles recommendation anyway not to use default service accounts). So you wll definitely be able to replicate this by deleting the service account, why I think this SA completely missing is related to the original report in this issue is that I am sure there is some situation where also the disabling will cause an error since the default SA is hard coded in the code so it most likely use it for some situations otherwise dont see why the check is even made that the service account exists if it would not be used for something. So since our codebase wont trigger the disabling issue can not reproduce that, but as said there definitely must be a situation where that wold be an issue since the tooling expects and checks for that service account existing.

If you look at the firebase-tools code it checks for existence of the compute engine service account not if its enabled or disabled, everything works for us if the service account is disabled (in those project we have the service account), but if you delete (or its not created when the API is enabled) the default Compute Engine service account then its not possible to deploy gen2 cloud functions. Since the configured service account for the functions has no effect on what checks the firebase-tools code does since it uses hardcoded compute engine service account name. Even if I modify the firebase-tools code to use custom name in place of those hardcoded compute service accounts the deploy fails to error from cloud side that the default service account is missing so atm we can not even hack the code to deploy the functions.

joehan commented 11 months ago

Hey @jketcham - since you tried to change this on the firebase-tools side and still ran into errors, could you open up an issue for this in the Cloud Functions public issue tracker? https://issuetracker.google.com/issues/new?component=187195&template=0

google-oss-bot commented 11 months ago

Hey @jketcham. We need more information to resolve this issue but there hasn't been an update in 7 weekdays. I'm marking the issue as stale and if there are no new updates in the next 3 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

google-oss-bot commented 11 months ago

Since there haven't been any recent updates here, I am going to close this issue.

@jketcham if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.

firebase / firebase-tools