search

akka / alpakka

Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.
https://doc.akka.io/docs/alpakka/current/
Other
1.26k stars 646 forks source link

Use google/cloud-sdk emulator to add integration tests for Google Cloud Storage #2168

Open seglo opened 4 years ago

seglo commented 4 years ago

There's no integration testing currently done in the google-cloud-stroage connector. This issue was raised in https://github.com/akka/alpakka/issues/2163 .

Some posts about using the google/cloud-sdk Docker image in a test suite:

https://medium.com/@mrkrijgsman/integration-testing-pub-sub-interactions-in-your-application-45e35476de23 https://medium.com/google-cloud/using-google-cloud-emulators-for-integration-tests-7812890ebe0d

lustefaniak commented 3 years ago

I don't think there is Cloud Storage emulator available.

Available groups for gcloud beta emulators:

      bigtable                *(BETA)*  Manage your local Bigtable emulator.
      datastore               *(BETA)*  Manage your local datastore emulator.
      firestore               *(BETA)*  Manage your local firestore emulator.
      pubsub                  *(BETA)*  Manage your local pubsub emulator.
      spanner                 *(BETA)*  Manage your local Spanner emulator.
seglo commented 3 years ago

I noticed cloud storage emulation wasn't mentioned in the image's github repo either. https://github.com/GoogleCloudPlatform/cloud-sdk-docker

Since you may have interest in this issue, I'm curious if you have any ideas to make our current testing better.

armanbilge commented 3 years ago

I think the best option is to use Hoverfly or similar to record a live session and then replay the recording for tests. I had a good experience using it for the updated BigQuery connector. There's even already a GCStorageStreamIntegrationSpec with comments describing how to run it against a live service account.

seglo commented 3 years ago

That sounds like a good option.