open-telemetry / opentelemetry-python-contrib

OpenTelemetry instrumentation for Python modules
https://opentelemetry.io
Apache License 2.0
647 stars 536 forks source link

flask instrumentation flaky test #2596

Open emdneto opened 2 weeks ago

emdneto commented 2 weeks ago

instrumentations-0 (pypy3, flask, ubuntu-20.04)


=================================== FAILURES ===================================
_______________ TestProgrammatic.test_flask_metrics_new_semconv ________________

self = <tests.test_programmatic.TestProgrammatic testMethod=test_flask_metrics_new_semconv>

    def test_flask_metrics_new_semconv(self):
        start = default_timer()
        self.client.get("/hello/123")
        self.client.get("/hello/321")
        self.client.get("/hello/756")
        duration = max(round((default_timer() - start) * 1000), 0)
        metrics_list = self.memory_metrics_reader.get_metrics_data()
        number_data_point_seen = False
        histogram_data_point_seen = False
        self.assertTrue(len(metrics_list.resource_metrics) != 0)
        for resource_metric in metrics_list.resource_metrics:
            self.assertTrue(len(resource_metric.scope_metrics) != 0)
            for scope_metric in resource_metric.scope_metrics:
                self.assertTrue(len(scope_metric.metrics) != 0)
                for metric in scope_metric.metrics:
                    self.assertIn(metric.name, _expected_metric_names_new)
                    data_points = list(metric.data.data_points)
                    self.assertEqual(len(data_points), 1)
                    for point in data_points:
                        if isinstance(point, HistogramDataPoint):
                            self.assertEqual(point.count, 3)
>                           self.assertAlmostEqual(
                                duration, point.sum, delta=10
                            )
E                           AssertionError: 17 != 0.011322596000013618 within 10 delta (16.988677403999986 difference)

Steps to reproduce Describe exactly how to reproduce the error. Include a code sample if applicable. Run tox -f pypy3-flask

What is the expected behavior? test pass everytime

What did you see instead? test is flaky

Additional context Add any other context about the problem here.

rahulhacker commented 1 week ago

@emdneto @xrmx Please assign the issue to me if the issue is still open

xrmx commented 1 week ago

@rahulhacker looking forward for your PR