False pass/fail reports to final-result.json

balena-os / leviathan

A distributed hardware testing framework

https://balena-os.github.io/leviathan

Apache License 2.0

22 stars 6 forks source link

False pass/fail reports to final-result.json #683

Open vipulgupta2048 opened 2 years ago

vipulgupta2048 commented 2 years ago

Report file generated was inaccurate by a long mile here: https://jenkins.product-os.io/job/leviathan-raspberrypi3/1119/artifact/client/build/reports/final-result.json

FD: https://www.flowdock.com/app/rulemotion/device-testing/threads/LahNc05_FWzqwWiPyJKE4SKZiqU Jenkins: https://jenkins.product-os.io/job/leviathan-raspberrypi3/1119/

vipulgupta2048 commented 2 years ago

Additionally https://jenkins.product-os.io/job/leviathan-fincm3/20/

vipulgupta2048 commented 2 years ago

Checked these out. My hypothesis:

RPi3 and fincm3 workers in the rig while running the latest client are still running old cores/workers release. Hence, the weird behavior we see for summary files. Keep this open if we see more.

vipulgupta2048 commented 2 years ago

Haven't heard back on this with more report closing