quarkiverse / quarkus-google-cloud-services

Google Cloud Services Quarkus Extensions
https://docs.quarkiverse.io/quarkus-google-cloud-services/main/index.html
Apache License 2.0
54 stars 33 forks source link

Add more flexibility for firestore emulator host. #630

Closed ctschubel closed 1 week ago

ctschubel commented 3 months ago

I would like to use the firestore emulator for testing my quarkus application. My issue is that the producer (to be precise the method io.quarkiverse.googlecloudservices.firestore.runtime.FirestoreProducer.useEmulatorCredentials()) only allows me to use a firestore emulator if 'quarkus.google.cloud.firestore.host-override' contains 'localhost'. I use gitlab-ci to run my tests and my build pipeline is configured to publish the firestore service not as 'localhost' but 'firestore' (firestore:9000). Something like:

...
services:
  - name: google/cloud-sdk
    alias: firestore
    pull_policy: always
    command:
    - gcloud
    - emulators
    - firestore
    - start
    - "--host-port=0.0.0.0:9000"

Its no big deal to overwrite that alias with localhost. It would just save me some configuration to allow other hosts as emulatorHosts as well. I use 'firestore' as standard alias for a while now and have some other apps written in SpringBoot that use the same ci configuration.

loicmathieu commented 3 months ago

You can do it by using quarkus.google.cloud.firestore.host-override.

ctschubel commented 3 months ago

Yes but only if the emulator host adress contains 'localhost'. See: useEmulatorCredentials.

If my firestore emulator is not accessible under localhost, the FirestoreProducer will never recognize it as emulator. When the connection is not recognized as emulator authorization is not working properly and some operations like batch writes don't work.

The emulator logs stuff like:

[firestore] WARNING: Operation failed: Batch writes require admin authentication. [firestore] com.google.cloud.datastore.core.exception.DatastoreException: Batch writes require admin authentication.

The problem is not that easy to see because some operations actually work and the errors are only shown in the logs of the emulator. It took me some time to find out whats going on..

loicmathieu commented 2 months ago

@jfbenckhuijsen can you explain why you restrict emulator credential to localhost and if we can safely relax it?

jfbenckhuijsen commented 2 months ago

Basically because the Firestore library itself has the same logic internally. Ie no real reason for us not to relax it, apart from checking what the impact would be in Firestore itself.

Can dive into that in about 2-3 weeks time due to holidays if that’s fitting

jfbenckhuijsen commented 2 months ago

Other reason btw is how to detect if it is an emulator running. If you run your application on something else than GCP I think you need to specify a host explicitly. Would need to validate that, but if that’s the case, we would need a flag to specifically trigger the emulator behavior. We could keep the localhost implies emulator == true automatically we now have.

loicmathieu commented 2 months ago

@jfbenckhuijsen thanks for your feedback and enjoy your holliday, we can definitely wait a few weeks ;)

Would need to validate that, but if that’s the case, we would need a flag to specifically trigger the emulator behavior. We could keep the localhost implies emulator == true automatically we now have.

This seems like a good plan: do as we do today with automatic behavior but allow manual configuration in case the automatic behavior is not what is needed.

jfbenckhuijsen commented 2 months ago

PR #644 created as an initial idea. No unit-tests yet to validate the behavior, need to check how we can easily test this.

ctschubel commented 1 week ago

Now everything works as expected. Thanks for your help! 😁 I'm closing the issue.