search

pendulum-chain / spacewalk-testing-service

GNU General Public License v3.0
0 stars 0 forks source link

Service gets stuck with tests running infinitely and does not start new tests. #14

Closed gianfra-t closed 7 months ago

gianfra-t commented 7 months ago

Error Description

The test service shows in the logs for some testing configurations Test already running for vault ..... at stage request_issue_completed even after well passed the maximum delay time of the vault to complete. These tests, successful or not, should have been removed from the testStages list and should have allowed others to start.

The error StellarAccountError: Unknown stellar account error was seen just before this behavior started.

Context

As soon as any test instance starts, the service will push this instance into a state variable testStages. If for some reason the scheduler attempts to start a test with the same configuration while the previous one is still ongoing, it will just ignore the start of the new one. After some maximum time the redeem/issue request will time out, returning an error which will stop the test and remove it from the state.

Possible causes and fixes

Somehow the StellarAccountError: Unknown stellar account error error may have ended the test but withouth it going through the removal process, or it could have created a never ending process that is currently not checked by any timer.

We could try to replicate this error (which probably occurred here), and to fix it we should ensure that it produces an error immediately which triggers the end of the test.

prayagd commented 7 months ago

Hey team! Please add your planning poker estimate with Zenhub @b-yap @bogdanS98 @ebma @gianfra-t @TorstenStueber