search

oktetlabs / test-environment

OKTET Labs Test Environment
Other
5 stars 10 forks source link

engine/tester: emit argument hash for all iterations #14

Closed okt-galaktionov closed 1 year ago

okt-galaktionov commented 1 year ago

When a test is run as part of a prologue, it does not get a TIN, and therefore does not emit its argument hash. This prevents some tools from telling different iterations of this test apart from each other.

Fix this problem by attaching the argument hash whenever it's possible.

Testing done: HTML logs now show argument hashes for prologue tests, bublik displays prologue arguments correctly now

ol-alexandra commented 1 year ago

I guess the idea is to distinguish different prologue failures in bublik. I think it is very useful. CC: @ol-damirm

okt-galaktionov commented 1 year ago

Sorry, I should've been more explicit. Without this patch, bublik can't properly show prologue arguments if the package/session is run multiple times with different arguments. Bublik always shows the arguments associated with the first such iteration, because the argument hashes are the same (read, null) in all prologue iterations, so bublik assumes that the arguments are the same as well.

While the issue could've been fixed entirely on bublik's side, I don't see any reason why prologues can't provide these argument hashes.