rabbitmq / rabbitmq-dotnet-client

RabbitMQ .NET client for .NET Standard 2.0+ and .NET 4.6.2+
https://www.rabbitmq.com/dotnet.html
Other
2.06k stars 574 forks source link

Separate out Unit, Integration and Parallel Integration tests #1409

Closed lukebakken closed 8 months ago

lukebakken commented 8 months ago

Start by moving some projects around

stebet commented 8 months ago

As an idea. I once experimented with moving all the unit tests that affect everyone (like server restarts, backoff thresholds) ti a separate project so they don't run at the same time as "normal" tests. That allowed me to run the bulk of the tests in parallel and then all the "server-manipulating" ones later, which cut down the test time considerably.

Might be something to consider.

michaelklishin commented 8 months ago

@stebet that's how we split unit and integration tests in many sub-projects related to RabbitMQ, in particular all the Common Test suites in RabbitMQ itself. CT supports parallel runs.

Note that if an integration test relies on any inherent shared state it'd have to be moved to the "sequential" group regardless of whether it manipulates nodes.

lukebakken commented 8 months ago

As an idea. I once experimented with moving all the unit tests that affect everyone (like server restarts, backoff thresholds) ti a separate project so they don't run at the same time as "normal" tests.

@stebet yep that's my plan! I want to have tests grouped into the appropriate projects so I can run them as parallel github actions workflows.

lukebakken commented 8 months ago

@stebet @michaelklishin well, I've tried to get the Test.Integration tests to run using Xunit's parallel feature, but there are flaky test results. This is most likely indicative of bugs in the test suite, or the library itself. My guess is that it has to do with mixing sync / async code at times.

What I'm going to do for now is force those tests to be sequential so that CI passes. Then, merge this PR and #1347 so that there are no long-running branches.

Then, I will figure out what's up with the test suite not being parallel-izable.