search

assertj / assertj-swing

Fluent assertions for Swing apps
Other
107 stars 52 forks source link

Random (difficult to reproduce) test failures #240

Closed ivan94fi closed 4 years ago

ivan94fi commented 5 years ago

Hi everyone. As always let me thank all the developers who are keeping this project up. Your work is really appreciated.

Now for our issue. As stated in the title, we occasionally run into problems in running tests of some components: the failures are connected to ComponentLookup exceptions and WaitTimedOutError errors appearing in both our Ubuntu 18.04 laptops and on Travis (ubuntu trusty worker).

There are mostly two cases when errors arise:

There are more details on this in our stackoverflow question.

As no one answered our question, we thought of opening an issue here, also because we followed the documented procedure to test.

The failing code is linked in the stackoverflow question.

Feel free to close this issue if you think it is a problem in our code and ask us if you need more details.

Thank you for your time

croesch commented 4 years ago

Hi @ivan94fi

sorry for not answering until now.

No, I don't think it's a problem of your code. There is still some kind of bug(s) in the assertj-swing-code. You can do two four things:

  1. Make sure to always use the latest version of AssertJ-Swing (because sometimes we increase stability ;-) )
  2. Start hunting these instability issues in AssertJ-Swing and file a PR with the fixes :-) Might be rewarded.
  3. Wait for somebody else to increase the stability. Might last until forever .. sorry
  4. Work around the instability. I think this is the only solution that works quickly.

Closing this as a duplicate of #230 #157 and #30

Your information is appreciated but I don't feel like more issues for the same topic is increasing the chance of fixing it ;-) the test suite in assertj-swing itself represents this test stability - so I'm aware of it every time I run tests on that project.

Reopen if you disagree :-)

ivan94fi commented 4 years ago

Hi @croesch

thanks for the reply.

In the end we were able to fix the instabilities by executing the tests in a vnc server (tightVNC), as described in AssertJ Swing docs.

Hope this can be useful for someone else having the same issues.