prashant-ramcharan / courgette-jvm

Multiprocess | Parallel Cucumber-JVM | Parallelize your Java Cucumber tests on a feature level or on a scenario level.
MIT License
130 stars 38 forks source link

On using thread count as more than 6,maxmium cases fail but on running on thread count as 4 or less than that then cases passes. #326

Closed s-kum-1 closed 2 years ago

s-kum-1 commented 2 years ago

I am using courgette-jvm dependency to execute parallel cases. On using thread count as more than 6, maximum cases fail but on running on thread count as 4 or less than that then cases pass. Kindly take a look.

prashant-ramcharan commented 2 years ago

Hi, could you share more details please?

  1. What errors do you see when you run 6 threads or more?
  2. Is any of the errors related to Courgette-JVM (i.e you see Courgette exceptions) ?
  3. Do you get the same test failure every-time?

Could it be that you have some shared stated in your test framework that fails because of parallel testing? You need to ensure all tests can pass in its own right and not have any shared state between tests.

If your Cucumber tests fails then Courgette will just report the failure, it does not change the statuses of test results.

s-kum-1 commented 2 years ago

Hi,

  1. What errors do you see when you run 6 threads or more? When we run on 6 threads or more then maximum time application does not loads fully which results into test case failure. Due to which new browser invokes. But we execute using 4 threads then application loads properly and it works fine.

  2. Is any of the errors related to Courgette-JVM (i.e you see Courgette exceptions) ? I do not see any Courgette exceptions

  3. Do you get the same test failure every-time? Loading issue appears on 6 or more threads so same failure appears on different test cases every time. But individually all tests can pass in its own right.

prashant-ramcharan commented 2 years ago

Please share a project to replicate this as it seems very specific to your test environment / the way your project is set up.

prashant-ramcharan commented 2 years ago

Closing inactive issue.