Stress test seems flaky

[hackaugusto]

https://circleci.com/gh/raiden-network/raiden/76628#tests/containers/18

[LefterisJP]

So should we skip it again?

[jomuel]

Currently skipped again (PR #5231)

[hackaugusto]

https://circleci.com/gh/raiden-network/raiden/89533#tests/containers/11

[LefterisJP]

This is one of our most complicated and complete integration tests. It is currently skipped. No wonder we find problems only during the SP run.

[jomuel]

This is a performance issue: The reason for the test failures are expired transfers. As a quick fix, we will try to relax the reveal_timeout parameter. In the long run, it may make sense to treat this test differently, as it is a stress test under very unrealistic circumstances (very short block time, multiple nodes plus synapse server and eth node running on the same machine, with in-memory databases.)

Seeing that, we might consider

• taking the stress test out of the integration tests into a circle job of its own, with higher ram/cpu power settings

• using a longer block time for it

• trying to run it with changed synapse settings (e. g. log level, synapse is very verbose when set to debug and it may affect performance)

I will leave the issue open for discussion.

[LefterisJP]

As discussed over RC increasing timeouts or block times is not a proper solution as it would just circumvent and possibly hide the problem.

Seeing as the stress test used to pass for the same settings some time in the past we have to actually investigate what changed and what has slowed the transfers down that much.