search

open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
3.02k stars 2.33k forks source link

[internal/otelarrow] Flaky test disabled: TestIntegrationMemoryLimited #34719

Open pjanotti opened 2 months ago

pjanotti commented 2 months ago

Component(s)

internal/otelarrow

Describe the issue you're reporting

Hit on #34358 see https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/10203882356/job/28231140032?pr=34358#step:6:518

=== FAIL: test TestIntegrationSelfTracing (11.03s)
    e2e_test.go:369: 
            Error Trace:    D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:369
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:205
                                        C:/hostedtoolcache/windows/go/1.21.12/x64/src/runtime/asm_amd64.s:1650
            Error:          Received unexpected error:
                            rpc error: code = Canceled desc = send wait: context deadline exceeded
            Test:           TestIntegrationSelfTracing
    e2e_test.go:369: 
            Error Trace:    D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:369
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:205
                                        C:/hostedtoolcache/windows/go/1.21.12/x64/src/runtime/asm_amd64.s:1650
            Error:          Received unexpected error:
                            rpc error: code = Canceled desc = send wait: context deadline exceeded
            Test:           TestIntegrationSelfTracing
    e2e_test.go:369: 
            Error Trace:    D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:369
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:205
                                        C:/hostedtoolcache/windows/go/1.21.12/x64/src/runtime/asm_amd64.s:1650
            Error:          Received unexpected error:
                            rpc error: code = Canceled desc = send wait: context deadline exceeded
            Test:           TestIntegrationSelfTracing
    e2e_test.go:369: 
            Error Trace:    D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:369
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:205
                                        C:/hostedtoolcache/windows/go/1.21.12/x64/src/runtime/asm_amd64.s:1650
            Error:          Received unexpected error:
                            rpc error: code = Canceled desc = send wait: context deadline exceeded
            Test:           TestIntegrationSelfTracing
    e2e_test.go:369: 
            Error Trace:    D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:369
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:205
                                        C:/hostedtoolcache/windows/go/1.21.12/x64/src/runtime/asm_amd64.s:1650
            Error:          Received unexpected error:
                            rpc error: code = Canceled desc = send wait: context deadline exceeded
            Test:           TestIntegrationSelfTracing
    e2e_test.go:369: 
            Error Trace:    D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:369
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:205
                                        C:/hostedtoolcache/windows/go/1.21.12/x64/src/runtime/asm_amd64.s:1650
            Error:          Received unexpected error:
                            rpc error: code = Canceled desc = send wait: context deadline exceeded
            Test:           TestIntegrationSelfTracing
    e2e_test.go:369: 
            Error Trace:    D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:369
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:205
                                        C:/hostedtoolcache/windows/go/1.21.12/x64/src/runtime/asm_amd64.s:1650
            Error:          Received unexpected error:
                            rpc error: code = Canceled desc = send wait: context deadline exceeded
            Test:           TestIntegrationSelfTracing
    e2e_test.go:369: 
            Error Trace:    D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:369
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:205
                                        C:/hostedtoolcache/windows/go/1.21.12/x64/src/runtime/asm_amd64.s:1650
            Error:          Received unexpected error:
                            rpc error: code = Canceled desc = send wait: context deadline exceeded
            Test:           TestIntegrationSelfTracing
    e2e_test.go:369: 
            Error Trace:    D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:369
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:205
                                        C:/hostedtoolcache/windows/go/1.21.12/x64/src/runtime/asm_amd64.s:1650
            Error:          Received unexpected error:
                            rpc error: code = Canceled desc = send wait: context deadline exceeded
            Test:           TestIntegrationSelfTracing
    e2e_test.go:369: 
            Error Trace:    D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:369
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:205
                                        C:/hostedtoolcache/windows/go/1.21.12/x64/src/runtime/asm_amd64.s:1650
            Error:          Received unexpected error:
                            rpc error: code = Canceled desc = send wait: context deadline exceeded
            Test:           TestIntegrationSelfTracing
    e2e_test.go:272: 
            Error Trace:    D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:272
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:418
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:220
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:476
            Error:          Not equal: 
                            expected: 10000
                            actual  : 4664
            Test:           TestIntegrationSelfTracing
github-actions[bot] commented 2 months ago

Pinging code owners:

pjanotti commented 2 months ago

Another hit at https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/10391822945/job/28775650790?pr=34656#step:6:507

pjanotti commented 2 months ago

Since both hits are on Windows /label os:windows

crobert-1 commented 2 months ago

Since both hits are on Windows /label os:windows

FYI: To add a label using automation, the /label message has to be at the beginning of the comment. Source

pjanotti commented 2 months ago

+1 https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/10493207856/job/29066610495?pr=34791#step:6:526

jmacd commented 2 months ago

Will take a look.

jmacd commented 2 months ago

I would like to recommend #34794, and if that fails I'll be glad to disable the test on Windows. Without trying to fix this, I'm not sure how we'd ever resolve it.

pjanotti commented 2 months ago

Many test failures in Windows are due to the scheduling and the default time tick resolution being different than *nix. The sleep added in #34794 seems a reasonable try.

jmacd commented 2 months ago

I have added one Skip to this test, will leave this issue open.

songy23 commented 5 days ago

This is still happening in Windows CIs though with different messages https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/11370646308/job/31630914777

=== Failed
=== FAIL: test TestIntegrationMemoryLimited (37.46s)
make[2]: *** [../../Makefile.Common:131: test] Error 1
    e2e_test.go:100: 
            Error Trace:    D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:100
make[1]: *** [Makefile:200: internal/otelarrow] Error 2
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/otelarrowreceiver/internal/arrow/arrow.go:892
make[1]: *** Waiting for unfinished jobs....
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/otelarrowreceiver/internal/arrow/arrow.go:702
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/otelarrowreceiver/internal/arrow/arrow.go:682
                                        C:/hostedtoolcache/windows/go/1.22.8/x64/src/runtime/asm_amd64.s:1695
            Error:          "4.166666666s" is not less than "4.1333912s"
            Test:           TestIntegrationMemoryLimited

=== FAIL: test TestIntegrationMemoryLimited (re-run 1) (44.79s)
    e2e_test.go:100: 
            Error Trace:    D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/internal/otelarrow/test/e2e_test.go:100
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/otelarrowreceiver/internal/arrow/arrow.go:892
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/otelarrowreceiver/internal/arrow/arrow.go:702
                                        D:/a/opentelemetry-collector-contrib/opentelemetry-collector-contrib/receiver/otelarrowreceiver/internal/arrow/arrow.go:682
                                        C:/hostedtoolcache/windows/go/1.22.8/x64/src/runtime/asm_amd64.s:1695
            Error:          "4.166666666s" is not less than "3.9117612s"
            Test:           TestIntegrationMemoryLimited

DONE 2 runs, 50 tests, 2 failures in 178.112s
✓  . (1.129s)