search

camunda / zeebe-process-test

Testing project for Camunda Platform 8
40 stars 15 forks source link

Run tests in parallel #42

Open pihme opened 2 years ago

pihme commented 2 years ago

Description We could change our own config to run unit tests in parallel in multiple threads. This way we will see if any problems pop up when tests are run in parallel.

remcowesterhoud commented 2 years ago

I played around with this for a little bit. For now we cannot run in parallel at all. This is because EZE starts a GRPC server with a hardcoded port (26500).

Lets try to get this working after we've copied EZE.

simara-svatopluk commented 2 years ago

Hello, I played with parallel test as well

And when I have at lest two test functions in a single test class, I randomly get java.util.ConcurrentModificationException.

After couple of hours of playing with it, I guess the problem is in RecordStreamSourceImpl, method records(). Although it creates unmodifiableList, the underlying list is mutable. And because the RecordStream is shared between test methods, it happens that one test writes into the records, and one iterates over it.

Anyway this is just a guess I wasn't able to confirm :-/

remcowesterhoud commented 2 years ago

The problem in spring-zeebe-test is a bit different from the one in zeebe-process-test. In Spring Zeebe the lifecycle of the engine is different from the lifecycle we use here. In Spring Zeebe one engine is started per test class, whereas here start a new engine for each test case in ZPT. There is an open issue for this in Spring Zeebe to align this: https://github.com/camunda-community-hub/spring-zeebe/issues/189

This is why you are running into the ConcurrentModificationException. Ideally we start a different engine for each test case so we can guarantee the tests are not influencing each other. This would mean each case has it's own RecordStreamSourceImpl which would prevent the exception you're seeing.

simara-svatopluk commented 2 years ago

Thanks for explaining the difference!

saig0 commented 3 months ago

[!Note] Heads up! We are building a new Java testing library for Camunda 8.6. The new library will replace Zeebe Process Test. Read more about upcoming changes here and stay tuned for updates. :rocket:

simara-svatopluk commented 3 months ago

Looking forward for that! Thanks for maintaining this project.