search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.11k stars 3.81k forks source link

sql: TestStreamerTightBudget failed #119675

Open cockroach-teamcity opened 8 months ago

cockroach-teamcity commented 8 months ago

sql.TestStreamerTightBudget failed on master @ 7bb52a7d1c75d5adfdfa53e5fcff6f5e6497408f:

                            ----
                            planning time: 302µs
                            execution time: 7ms
                            distribution: full
                            vectorized: true
                            rows decoded from KV: 10 (5.0 MiB, 10 gRPC calls)
                            cumulative time spent in KV: 6ms
                            maximum memory usage: 5.3 MiB
                            network usage: 0 B (0 messages)
                            sql cpu time: 5ms
                            isolation level: serializable
                            priority: normal
                            quality of service: regular

                            • group (scalar)
                            │ nodes: n1
                            │ actual row count: 1
                            │ sql cpu time: 23µs
                            │
                            └── • render
                                │
                                └── • index join (streamer)
                                    │ nodes: n1
                                    │ actual row count: 5
                                    │ KV time: 6ms
                                    │ KV contention time: 0µs
                                    │ KV rows decoded: 5
                                    │ KV bytes read: 5.0 MiB
                                    │ KV gRPC calls: 9
                                    │ estimated max memory allocated: 4.6 MiB
                                    │ estimated max sql temp disk usage: 0 B
                                    │ sql cpu time: 5ms
                                    │ table: t@t_pkey
                                    │
                                    └── • scan
                                          nodes: n1
                                          actual row count: 5
                                          KV time: 115µs
                                          KV contention time: 0µs
                                          KV rows decoded: 5
                                          KV bytes read: 168 B
                                          KV gRPC calls: 1
                                          estimated max memory allocated: 10 KiB
                                          sql cpu time: 6µs
                                          missing stats
                                          table: t@t_k_idx
                                          spans: [/1 - /1]
    panic.go:523: -- test log scope end --
test logs left over in: outputs.zip/logTestStreamerTightBudget1878394043
--- FAIL: TestStreamerTightBudget (0.54s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/sql-queries

This test on roachdash | Improve this report!

Jira issue: CRDB-36274

yuzefovich commented 8 months ago

I got a failure on this run. Attaching the bundle. bundle.zip

cockroach-teamcity commented 8 months ago

sql.TestStreamerTightBudget failed on master @ ebf92d22a7e4afe2423e681d49c615104b5f0098:

        │ vectorized batch count: 1
        │ sql cpu time: 18µs
        │ estimated row count: 1 (missing stats)
        │ aggregate 0: sum(column6)
        │
        └── • render
            │ columns: (column6 int)
            │ render column6: (length((blob)[string]))[int]
            │
            └── • index join (streamer)
                │ columns: (k int, blob string)
                │ nodes: n1
                │ actual row count: 5
                │ vectorized batch count: 5
                │ KV time: 8ms
                │ KV contention time: 0µs
                │ KV rows decoded: 5
                │ KV pairs read: 5
                │ KV bytes read: 5.0 MiB
                │ KV gRPC calls: 9
                │ estimated max memory allocated: 2.2 MiB
                │ estimated max sql temp disk usage: 0 B
                │ sql cpu time: 5ms
                │ MVCC step count (ext/int): 0/0
                │ MVCC seek count (ext/int): 9/9
                │ estimated row count: 10 (missing stats)
                │ table: t@t_pkey
                │ key columns: pk
                │
                └── • scan
                      columns: (pk int, k int)
                      nodes: n1
                      actual row count: 5
                      vectorized batch count: 3
                      KV time: 499µs
                      KV contention time: 0µs
                      KV rows decoded: 5
                      KV pairs read: 5
                      KV bytes read: 140 B
                      KV gRPC calls: 1
                      estimated max memory allocated: 20 KiB
                      sql cpu time: 18µs
                      MVCC step count (ext/int): 5/5
                      MVCC seek count (ext/int): 1/1
                      estimated row count: 10 (missing stats)
                      table: t@t_k_idx
                      spans: /1-/2
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestStreamerTightBudget1381126795
--- FAIL: TestStreamerTightBudget (0.50s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

cockroach-teamcity commented 7 months ago

sql.TestStreamerTightBudget failed on master @ 7dfff9430b0aedaac5bb57e06d704a527336863e:

        │ vectorized batch count: 1
        │ sql cpu time: 24µs
        │ estimated row count: 1 (missing stats)
        │ aggregate 0: sum(column6)
        │
        └── • render
            │ columns: (column6 int)
            │ render column6: (length((blob)[string]))[int]
            │
            └── • index join (streamer)
                │ columns: (k int, blob string)
                │ nodes: n1
                │ actual row count: 5
                │ vectorized batch count: 5
                │ KV time: 10ms
                │ KV contention time: 0µs
                │ KV rows decoded: 5
                │ KV pairs read: 5
                │ KV bytes read: 5.0 MiB
                │ KV gRPC calls: 9
                │ estimated max memory allocated: 2.5 MiB
                │ estimated max sql temp disk usage: 0 B
                │ sql cpu time: 6ms
                │ MVCC step count (ext/int): 0/0
                │ MVCC seek count (ext/int): 9/9
                │ estimated row count: 10 (missing stats)
                │ table: t@t_pkey
                │ key columns: pk
                │
                └── • scan
                      columns: (pk int, k int)
                      nodes: n1
                      actual row count: 5
                      vectorized batch count: 3
                      KV time: 572µs
                      KV contention time: 0µs
                      KV rows decoded: 5
                      KV pairs read: 5
                      KV bytes read: 168 B
                      KV gRPC calls: 1
                      estimated max memory allocated: 10 KiB
                      sql cpu time: 8µs
                      MVCC step count (ext/int): 5/5
                      MVCC seek count (ext/int): 1/1
                      estimated row count: 10 (missing stats)
                      table: t@t_k_idx
                      spans: /1-/2
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestStreamerTightBudget1756274810
--- FAIL: TestStreamerTightBudget (0.74s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

cockroach-teamcity commented 6 months ago

sql.TestStreamerTightBudget failed on master @ e90d15c9807775a1a1d89f75cac4c9383c325709:

        │ vectorized batch count: 1
        │ sql cpu time: 19µs
        │ estimated row count: 1 (missing stats)
        │ aggregate 0: sum(column6)
        │
        └── • render
            │ columns: (column6 int)
            │ render column6: (length((blob)[string]))[int]
            │
            └── • index join (streamer)
                │ columns: (k int, blob string)
                │ nodes: n1
                │ actual row count: 5
                │ vectorized batch count: 5
                │ KV time: 10ms
                │ KV contention time: 0µs
                │ KV rows decoded: 5
                │ KV pairs read: 5
                │ KV bytes read: 5.0 MiB
                │ KV gRPC calls: 9
                │ estimated max memory allocated: 3.8 MiB
                │ estimated max sql temp disk usage: 0 B
                │ sql cpu time: 6ms
                │ MVCC step count (ext/int): 0/0
                │ MVCC seek count (ext/int): 9/9
                │ estimated row count: 10 (missing stats)
                │ table: t@t_pkey
                │ key columns: pk
                │
                └── • scan
                      columns: (pk int, k int)
                      nodes: n1
                      actual row count: 5
                      vectorized batch count: 3
                      KV time: 494µs
                      KV contention time: 0µs
                      KV rows decoded: 5
                      KV pairs read: 5
                      KV bytes read: 168 B
                      KV gRPC calls: 1
                      estimated max memory allocated: 10 KiB
                      sql cpu time: 7µs
                      MVCC step count (ext/int): 5/5
                      MVCC seek count (ext/int): 1/1
                      estimated row count: 10 (missing stats)
                      table: t@t_k_idx
                      spans: /1-/2
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestStreamerTightBudget268233098
--- FAIL: TestStreamerTightBudget (0.56s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

cockroach-teamcity commented 6 months ago

sql.TestStreamerTightBudget failed on master @ 2f498d0abcf4130e5c872b73200086d6a3af9f02:

        │ vectorized batch count: 1
        │ sql cpu time: 33µs
        │ estimated row count: 1 (missing stats)
        │ aggregate 0: sum(column6)
        │
        └── • render
            │ columns: (column6 int)
            │ render column6: (length((blob)[string]))[int]
            │
            └── • index join (streamer)
                │ columns: (k int, blob string)
                │ nodes: n1
                │ actual row count: 5
                │ vectorized batch count: 5
                │ KV time: 18ms
                │ KV contention time: 0µs
                │ KV rows decoded: 5
                │ KV pairs read: 5
                │ KV bytes read: 5.0 MiB
                │ KV gRPC calls: 9
                │ estimated max memory allocated: 3.1 MiB
                │ estimated max sql temp disk usage: 0 B
                │ sql cpu time: 9ms
                │ MVCC step count (ext/int): 0/0
                │ MVCC seek count (ext/int): 9/9
                │ estimated row count: 10 (missing stats)
                │ table: t@t_pkey
                │ key columns: pk
                │
                └── • scan
                      columns: (pk int, k int)
                      nodes: n1
                      actual row count: 5
                      vectorized batch count: 3
                      KV time: 522µs
                      KV contention time: 0µs
                      KV rows decoded: 5
                      KV pairs read: 5
                      KV bytes read: 168 B
                      KV gRPC calls: 1
                      estimated max memory allocated: 10 KiB
                      sql cpu time: 7µs
                      MVCC step count (ext/int): 5/5
                      MVCC seek count (ext/int): 1/1
                      estimated row count: 10 (missing stats)
                      table: t@t_k_idx
                      spans: /1-/2
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestStreamerTightBudget2379595889
--- FAIL: TestStreamerTightBudget (0.60s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

cockroach-teamcity commented 6 months ago

sql.TestStreamerTightBudget failed on master @ 30c8f44195d8440b112e3c927d49ea193519afe8:

        │ vectorized batch count: 1
        │ sql cpu time: 22µs
        │ estimated row count: 1 (missing stats)
        │ aggregate 0: sum(column6)
        │
        └── • render
            │ columns: (column6 int)
            │ render column6: (length((blob)[string]))[int]
            │
            └── • index join (streamer)
                │ columns: (k int, blob string)
                │ nodes: n1
                │ actual row count: 5
                │ vectorized batch count: 5
                │ KV time: 10ms
                │ KV contention time: 0µs
                │ KV rows decoded: 5
                │ KV pairs read: 5
                │ KV bytes read: 5.0 MiB
                │ KV gRPC calls: 9
                │ estimated max memory allocated: 2.5 MiB
                │ estimated max sql temp disk usage: 0 B
                │ sql cpu time: 5ms
                │ MVCC step count (ext/int): 0/0
                │ MVCC seek count (ext/int): 9/9
                │ estimated row count: 10 (missing stats)
                │ table: t@t_pkey
                │ key columns: pk
                │
                └── • scan
                      columns: (pk int, k int)
                      nodes: n1
                      actual row count: 5
                      vectorized batch count: 3
                      KV time: 547µs
                      KV contention time: 0µs
                      KV rows decoded: 5
                      KV pairs read: 5
                      KV bytes read: 140 B
                      KV gRPC calls: 1
                      estimated max memory allocated: 20 KiB
                      sql cpu time: 17µs
                      MVCC step count (ext/int): 5/5
                      MVCC seek count (ext/int): 1/1
                      estimated row count: 10 (missing stats)
                      table: t@t_k_idx
                      spans: /1-/2
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestStreamerTightBudget3758826815
--- FAIL: TestStreamerTightBudget (0.49s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

cockroach-teamcity commented 6 months ago

sql.TestStreamerTightBudget failed on master @ 0144c31513c1ae5cdef4d78a051c17e40813f34d:

        │ vectorized batch count: 1
        │ sql cpu time: 32µs
        │ estimated row count: 1 (missing stats)
        │ aggregate 0: sum(column6)
        │
        └── • render
            │ columns: (column6 int)
            │ render column6: (length((blob)[string]))[int]
            │
            └── • index join (streamer)
                │ columns: (k int, blob string)
                │ nodes: n1
                │ actual row count: 5
                │ vectorized batch count: 5
                │ KV time: 18ms
                │ KV contention time: 0µs
                │ KV rows decoded: 5
                │ KV pairs read: 5
                │ KV bytes read: 5.0 MiB
                │ KV gRPC calls: 9
                │ estimated max memory allocated: 2.1 MiB
                │ estimated max sql temp disk usage: 0 B
                │ sql cpu time: 11ms
                │ MVCC step count (ext/int): 0/0
                │ MVCC seek count (ext/int): 9/9
                │ estimated row count: 10 (missing stats)
                │ table: t@t_pkey
                │ key columns: pk
                │
                └── • scan
                      columns: (pk int, k int)
                      nodes: n1
                      actual row count: 5
                      vectorized batch count: 3
                      KV time: 705µs
                      KV contention time: 0µs
                      KV rows decoded: 5
                      KV pairs read: 5
                      KV bytes read: 140 B
                      KV gRPC calls: 1
                      estimated max memory allocated: 20 KiB
                      sql cpu time: 40µs
                      MVCC step count (ext/int): 5/5
                      MVCC seek count (ext/int): 1/1
                      estimated row count: 10 (missing stats)
                      table: t@t_k_idx
                      spans: /1-/2
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestStreamerTightBudget144351213
--- FAIL: TestStreamerTightBudget (0.68s)

Parameters:

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

rytaft commented 5 months ago

@yuzefovich are you planning to work on this issue or should we put it back in Triage? It's now breached SLA

yuzefovich commented 5 months ago

I think it's rare enough to be downgraded to P-3. I do plan to look into this eventually (I cleared almost all long-standing issues that I've been assigned to, this might be the last one).

yuzefovich commented 3 months ago

I haven't had time to look closely into this, but I don't really have any hypothesis for why this test fails rarely, and the stmt bundle doesn't appear to provide any hints. I'll keep this assigned to myself and in the "bugs to fix" bucket given that the failures are quite rare (like once a month).