search

zarusz / SlimMessageBus

Lightweight message bus interface for .NET (pub/sub and request-response) with transport plugins for popular message brokers.
Apache License 2.0
467 stars 78 forks source link

[Integration testing] Containers for local development #262

Closed EtherZa closed 3 months ago

EtherZa commented 3 months ago

Testing if you aren't @zarusz can be a little painful with the test configurations tailored for private resources ;)

Please consider this PR. It allows for a full test coverage running everything locally except for Azure Service Bus and Azure Event Hub which don't have containers available.

To ease some of the integration, Kafka and Mqtt are both hosted without certs. It does require some minor changes to some of the bootstrapping for the integration tests, but won't have any impact (beyond configuration) for the build pipeline.

I understand that build pipeline will be broken with the change, but that should be easily resolved by updating the devops version of secrets.txt with the sample provided.

Local development environment using docker

  1. Copy src/secrets.txt.sample to src/secrets.txt
  2. Add connection strings for personal instances of the below to secrets.txt
    • Azure Service Bus
    • Azure Event Hub
  3. there is no auto-provisioning available for event hubs, so the below event hubs and consumer groups will need to be created manually (event hub => consumer group)
    • test-ping => subscriber
    • test-echo => handler
    • test-echo-resp => response-reader
  4. open a terminal to src/Infrastructure
  5. execute docker compose up --force-recreate -V or ./compose.sh
  6. Run tests and smile
zarusz commented 3 months ago

I agree that having test containers where possible is a step forward.

Let me think about this some more and perhaps accept this PR into a side master branch and then iterate on combining with the private cloud resources that are currently used for testing.

Again, thanks for your help and effort.

EtherZa commented 3 months ago

Again, thanks for your help and effort.

@zarusz It's a pleasure. We all win here

Let me think about this some more and perhaps accept this PR into a side master branch and then iterate on combining with the private cloud resources that are currently used for testing.

The first approach I took was to just add support for environment variables to BaseIntegrationTest.Configuratoin. Combining that with a .runsettings worked well until I tried to get it to work with live unit testing. Visual Studio ignores .runsettings in LUT.

This approach is still not perfect as secrets.txt is not included with the LUT shadow copy. Manually copying the file to the LUT directory gets the tests running successfully at least.

zarusz commented 3 months ago

@EtherZa I've adjusted the build.yaml and added the new secrets in GitHub to support int tests running against local and cloud infra. Please rebase against master.

Hopefully, it will work after one shot :)

EtherZa commented 3 months ago

Works like a charm - local tests are all green.

The Kafka and Event Hub tests intermittently need an extra run as there appears to be a race condition between each set; but that is a minor issue for now.

I am also hoping that you will forgive the code coverage complaint that SonarCloud is raising - it is the empty/commented line predicate in the secrets file reader. The build pipeline clearly makes use of neither condition :)

Thank you @zarusz. This will make a big difference.

sonarcloud[bot] commented 3 months ago

Quality Gate Failed Quality Gate failed

Failed conditions
45.5% Coverage on New Code (required ≥ 75%)

See analysis details on SonarCloud