search

paritytech / zombienet

A cli tool to easily spawn ephemeral Polkadot/Substrate networks and perform tests against them.
https://paritytech.github.io/zombienet/
GNU General Public License v3.0
157 stars 92 forks source link

Zombienet reports failure even though conditions are met #1652

Closed skunert closed 5 months ago

skunert commented 9 months ago

Test in question runs on CI: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4864909 Test file: https://github.com/paritytech/polkadot-sdk/blob/master/cumulus/zombienet/tests/0002-pov_recovery.zndsl

Zombienet reports:

 1/8/2024, 2:07:54 PM         │ ❌ three: count of log lines containing "Importing block retrieved using pov_recovery" is greater… │

It is omitted here, but from the test file we see that we should check that the log line should appear more than 19 times.

However, if I click the grafana link and search for "Importing block retrieved using pov_recovery" I find 22 hits in the time range.

image

So I would have expected this zombienet test to succeed.

Additional problem: When I download the logs from the artifacts in gitlab, the logs for node "three" are incomplete. They only cover the last minute or so of the run, even though grafana has more logs available. Not sure how that happens.

Faulty log here: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4864909/artifacts/external_file/zombienet-logs/three.log

pepoviola commented 9 months ago

Hi @skunert thanks for reporting, the root cause of this issue is that we are using kubectl to access the logs and could be that the logs from the host vm were rotated so the the logs we get from k8s are truncated. We have this https://github.com/paritytech/zombienet/issues/1565 tracking the fix, I will try to fix today and draft a new release asap.

Thx!!

skunert commented 9 months ago

Ah ok! Thanks for the fast response 👍 .

pepoviola commented 5 months ago

Hi @skunert, this should be already fixed. Closing here. Thx!