search

open-telemetry / opentelemetry-python

OpenTelemetry Python API and SDK
https://opentelemetry.io
Apache License 2.0
1.66k stars 568 forks source link

Skip test case when running in Windows or pypy #3986

Open ocelotl opened 1 week ago

ocelotl commented 1 week ago

Fixes #3985

xrmx commented 1 week ago

I would bump the delta to 30 instead

ocelotl commented 2 days ago

I would bump the delta to 30 instead

Next time it can fail with a delta of 31 :shrug: I don't see much sense in chasing this issue by increasing the delta, as far as we know this can fail with pretty much any delta but we are more certain that this happens only on Pypy or Windows.

xrmx commented 1 day ago

I would bump the delta to 30 instead

Next time it can fail with a delta of 31 🤷 I don't see much sense in chasing this issue by increasing the delta, as far as we know this can fail with pretty much any delta but we are more certain that this happens only on Pypy or Windows.

I think it would be nice to try to keep coverage on Windows I guess

ocelotl commented 1 day ago

I would bump the delta to 30 instead

Next time it can fail with a delta of 31 🤷 I don't see much sense in chasing this issue by increasing the delta, as far as we know this can fail with pretty much any delta but we are more certain that this happens only on Pypy or Windows.

I think it would be nice to try to keep coverage on Windows I guess

It would be great to have this test run on Windows, but it randomly fails there and that is out of our control. By the way, here here it failed with a delta of 34:

self.assertAlmostEqual((export_time - start_time) * 1e3, 500, delta=25)
E       AssertionError: 534.8496437072754 != 500 within 25 delta (34.84964370727539 difference)

So, either we skip this test case, or we automate what we always do, which is rerunning this test case everytime it fails.