Flaky Test: packetbeat/route TestDefault - `panic: runtime error: index out of range [3] with length 3`

belimawr commented 3 months ago

Flaky Test

Test Name: TestDefault
Link: https://github.com/elastic/beats/blob/43d5dd4342a9ff56f40fafe1ba2ad7ed0a03b6c3/packetbeat/route/route_windows_test.go#L63
Branch: main
Artifact Link:
Notes: It started failing consistently on main after https://github.com/elastic/beats/commit/62b4c18f2c13921814ee133f3ec4d44fe69ceba9 was merged. However this commit only removed two debug log lines from Filebeat. I cannot see any correlation between this commit and the Packetbeat test panic.

Stack Trace

2024-05-23 03:18:35 EDT === Failed
2024-05-23 03:18:35 EDT === FAIL: packetbeat/route TestDefault (0.87s)
2024-05-23 03:18:35 EDT panic: runtime error: index out of range [3] with length 3 [recovered]
2024-05-23 03:18:35 EDT     panic: runtime error: index out of range [3] with length 3
2024-05-23 03:18:35 EDT 
2024-05-23 03:18:35 EDT goroutine 6 [running]:
2024-05-23 03:18:35 EDT testing.tRunner.func1.2({0x899340, 0xc0000909f0})
2024-05-23 03:18:35 EDT     C:/Users/Buildkite/.go/go-1.21.10/src/testing/testing.go:1545 +0x238
2024-05-23 03:18:35 EDT testing.tRunner.func1()
2024-05-23 03:18:35 EDT     C:/Users/Buildkite/.go/go-1.21.10/src/testing/testing.go:1548 +0x397
2024-05-23 03:18:35 EDT panic({0x899340?, 0xc0000909f0?})
2024-05-23 03:18:35 EDT     C:/Users/Buildkite/.go/go-1.21.10/src/runtime/panic.go:914 +0x21f
2024-05-23 03:18:35 EDT github.com/elastic/beats/v7/packetbeat/route.defaultRoute(0xc0000c2ab7?)
2024-05-23 03:18:35 EDT     C:/buildkite-agent/builds/bk-agent-prod-gcp-1716448135293881985/elastic/beats-packetbeat/packetbeat/route/route_windows_test.go:63 +0xd72
2024-05-23 03:18:35 EDT github.com/elastic/beats/v7/packetbeat/route.TestDefault(0xc00003b520)
2024-05-23 03:18:35 EDT     C:/buildkite-agent/builds/bk-agent-prod-gcp-1716448135293881985/elastic/beats-packetbeat/packetbeat/route/route_test.go:36 +0x72
2024-05-23 03:18:35 EDT testing.tRunner(0xc00003b520, 0x8ba630)
2024-05-23 03:18:35 EDT     C:/Users/Buildkite/.go/go-1.21.10/src/testing/testing.go:1595 +0xff
2024-05-23 03:18:35 EDT created by testing.(*T).Run in goroutine 1
2024-05-23 03:18:35 EDT     C:/Users/Buildkite/.go/go-1.21.10/src/testing/testing.go:1648 +0x3ad

When I run the same tests on Linux I get an error instead of a panic:

tiago@millennium-falcon beats/packetbeat/route  v1.22.2  HEAD [$?] % go test ./...
--- FAIL: TestDefault (0.00s)
    route_test.go:43: unexpected error from Default(10): got:route: not found want:<nil>
    route_test.go:59: unexpected interface for family 10: got: want:wlp9s0
    route_test.go:63: unexpected interface for family 10: got:-1 want:3
FAIL
FAIL    github.com/elastic/beats/v7/packetbeat/route    0.036s
FAIL

I also tested it on my Windows machine and it passed, even with 100 repetitions.

elasticmachine commented 3 months ago

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

elasticmachine commented 3 months ago

Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform)

efd6 commented 3 months ago

It looks like this is a result of netsh now emitting version information and this not being properly handled by the helper since it was unknown behaviour.

https://buildkite.com/elastic/beats-packetbeat/builds/5344#018ff0f6-13bc-4375-84d8-5c434707c120/100-274

=== FAIL: packetbeat/route TestDefault (0.95s)
    route_test.go:38: unexpected error from defaultRoute(23): unexpected netsh  line: "netsh.exe: version 10.0.20348.1"

        Publish  Type      Met  Prefix                    Idx  Gateway/Interface Name
        -------  --------  ---  ------------------------  ---  ------------------------
        No       System    256  ::1/128                     1  Loopback Pseudo-Interface 1
        No       System    256  fe80::/64                   4  Ethernet
        No       System    256  fe80::/64                   9  vEthernet (nat)
        No       System    256  fe80::557:e168:e133:5781/128    9  vEthernet (nat)
        No       System    256  fe80::e58d:ff83:e072:e694/128    4  Ethernet
        No       System    256  ff00::/8                    1  Loopback Pseudo-Interface 1
        No       System    256  ff00::/8                    4  Ethernet
        No       System    256  ff00::/8                    9  vEthernet (nat)

        netsh.exe: version 10.0.20348.1

elastic / beats

Flaky Test: packetbeat/route TestDefault - `panic: runtime error: index out of range [3] with length 3` #39698

Flaky Test

Stack Trace