open-telemetry / opentelemetry-python-contrib

OpenTelemetry instrumentation for Python modules
https://opentelemetry.io
Apache License 2.0
647 stars 535 forks source link

Urllib instrumentation tox test fails: Unexpected count #2646

Open Charlie-lizhihan opened 2 days ago

Charlie-lizhihan commented 2 days ago

Describe your environment

OS: macOS with apple chip Python version: for this tox test, it will fail on every version

What happened?

When I run tox -e py39-test-instrumentation-urllib, I will get the error

______________________________________________________________________________________ TestUrllibMetricsInstrumentation.test_metric_uninstrument _______________________________________________________________________________________

self = <tests.test_metrics_instrumentation.TestUrllibMetricsInstrumentation testMethod=test_metric_uninstrument>

    @mark.skipif(
        python_implementation() == "PyPy", reason="Fails randomly in pypy"
    )
    def test_metric_uninstrument(self):
        with request.urlopen(self.URL):

            self.assertEqual(
                len(
                    (
                        self.memory_metrics_reader.get_metrics_data()
                        .resource_metrics[0]
                        .scope_metrics[0]
                        .metrics
                    )
                ),
                3,
            )

            self.assertEqual(
                (
                    self.memory_metrics_reader.get_metrics_data()
                    .resource_metrics[0]
                    .scope_metrics[0]
                    .metrics[0]
                    .data.data_points[0]
                    .bucket_counts[1]
                ),
                1,
            )
            self.assertEqual(
                (
                    self.memory_metrics_reader.get_metrics_data()
                    .resource_metrics[0]
                    .scope_metrics[0]
                    .metrics[1]
                    .data.data_points[0]
                    .bucket_counts[0]
                ),
                1,
            )
            self.assertEqual(
                (
                    self.memory_metrics_reader.get_metrics_data()
                    .resource_metrics[0]
                    .scope_metrics[0]
                    .metrics[2]
                    .data.data_points[0]
                    .bucket_counts[2]
                ),
                1,
            )

        with request.urlopen(self.URL):

            self.assertEqual(
                len(
                    (
                        self.memory_metrics_reader.get_metrics_data()
                        .resource_metrics[0]
                        .scope_metrics[0]
                        .metrics
                    )
                ),
                3,
            )

>           self.assertEqual(
                (
                    self.memory_metrics_reader.get_metrics_data()
                    .resource_metrics[0]
                    .scope_metrics[0]
                    .metrics[0]
                    .data.data_points[0]
                    .bucket_counts[1]
                ),
                2,
            )
E           AssertionError: 1 != 2

Steps to Reproduce

tox -e py39-test-instrumentation-urllib

Expected Result

all test pass

Actual Result

error occur

Additional context

But if try many times, it is possible pass, so it's pretty flaky

Would you like to implement a fix?

None