search

jancajthaml-openbank / e2e

rolling integration, contract and performance testing
Apache License 2.0
0 stars 0 forks source link

performance tests flake service was TERM #132

Closed jancajthaml closed 2 years ago

jancajthaml commented 3 years ago

Symptoms:

Traceback (most recent call last):
  File "/opt/app/perf/main.py", line 199, in <module>
    main()
  File "/opt/app/perf/main.py", line 126, in main
    manager.teardown()
  File "/opt/app/perf/appliance_manager.py", line 241, in teardown
    del self[name]
  File "/opt/app/perf/appliance_manager.py", line 192, in __delitem__
    node.teardown()
  File "/opt/app/perf/unit/vault_unit.py", line 42, in teardown
    eventual_teardown()
  File "/opt/app/perf/helpers/eventually.py", line 33, in __call__
    raise self.__last_exception
  File "/opt/app/perf/helpers/eventually.py", line 27, in __call__
    return self.__block(*args, **kwargs)
  File "/opt/app/perf/unit/vault_unit.py", line 40, in eventual_teardown
    assert code == 'OK', code + ' ' + str(result) + ' ' + str(error)
AssertionError: SIGTERM

Reason:

only lake is currently configured that it always stops and recovers from and with SIGTERM, vault and ledger are non deterministic and timing sensitive.

To fix:

Vault and Ledger needs update for their service definition, setup proper stop timeouts and stop exit codes.