MarathonLabs / marathon

Cross-platform test runner
https://docs.marathonlabs.io
GNU General Public License v2.0
584 stars 121 forks source link

[iOS only] Negative duration for some test executions #984

Open msd117c opened 6 days ago

msd117c commented 6 days ago

Describe the bug For a given (iOS) test, the test runner is writing a (huge) negative duration in the influxDB bucket we use for Marathon. Captura de pantalla 2024-11-20 a las 15 05 14 These durations belong to the same test.

To Reproduce It is hard to reproduce the issue because the data comes from executions during development and it is not the only one reporting data for that time window, making it harder to locate the logs.

Expected behavior I would expect a positive test duration or 0 if something unexpected happened, though that would introduce noise in the metrics (less than actual noise with that negative value though).

Logs and reports I can't access the logs but the test was reported as not successful to the influxDB instance (so I guess we can expect a greater than 0 duration).

Devices (please complete the following information):

Additional context I will try to manually reproduce the issue to have access to the logs. This only happened with an iOS setup (afaik Android never had this issue).

Malinskiy commented 6 days ago

The huge negative value is a negative unix epoch in milliseconds. duration = end - begin, so a huge negative value means that some specific test didn't report an end value. Without a reproducer, it's hard to debug such an issue. I'm happy to review a PR with a fix as well if you can't provide a reproducer.

Thanks!

msd117c commented 6 days ago

@Malinskiy I opened a draft PR trying to address the issue here. Let me know if you think that could solve the issue. I will try to reproduce the issue locally when possible and try the branch build to compare.