sql: crash for `explain analyze (debug)` - NPE in `generateMeta()`

[andreimatei]

Happens on both master and release-21.1. Some distribution of the query needs to be necessary.

./cockroach demo --nodes=9 --global
create table promos(id serial primary key, s string);
create table users(id uuid default gen_random_uuid() primary key, promo_id int references promos(id));
explain analyze (debug) insert into users (promo_id) values (642606224929619969);

* EXPLAIN ANALYZE (DEBUG) INSERT INTO users(promo_id) VALUES (642606224929619969)
*
*
* ERROR: [n1,client=127.0.0.1:38888,hostssl,user=demo] a panic has occurred!
* runtime error: invalid memory address or nil pointer dereference
* (1) attached stack trace
*   -- stack trace:
*   | github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn.func1
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/conn_executor.go:480
*   | [...repeated from below...]
* Wraps: (2) while executing: EXPLAIN ANALYZE (DEBUG) INSERT INTO _(_) VALUES (_)
* Wraps: (3) attached stack trace
*   -- stack trace:
*   | github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn.func1
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/conn_executor.go:480
*   | runtime.gopanic
*   |   /home/andrei/goroot/src/runtime/panic.go:969
*   | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError.func1
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/colexecerror/error.go:58
*   | runtime.gopanic
*   |   /home/andrei/goroot/src/runtime/panic.go:969
*   | runtime.panicmem
*   |   /home/andrei/goroot/src/runtime/panic.go:212
*   | runtime.sigpanic
*   |   /home/andrei/goroot/src/runtime/signal_unix.go:742
*   | github.com/cockroachdb/cockroach/pkg/kv.(*Txn).Type
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/kv/txn.go:1224
*   | github.com/cockroachdb/cockroach/pkg/sql/execinfra.GetLeafTxnFinalState
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/execinfra/base.go:267
*   | github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*joinReader).generateMeta
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/rowexec/joinreader.go:761
*   | github.com/cockroachdb/cockroach/pkg/sql/rowexec.newJoinReader.func1
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/rowexec/joinreader.go:287
*   | github.com/cockroachdb/cockroach/pkg/sql/execinfra.(*ProcessorBase).moveToTrailingMeta
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/execinfra/processorsbase.go:727
*   | github.com/cockroachdb/cockroach/pkg/sql/execinfra.(*ProcessorBase).DrainHelper
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/execinfra/processorsbase.go:646
*   | github.com/cockroachdb/cockroach/pkg/sql/rowexec.(*joinReader).Next
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/rowexec/joinreader.go:540
*   | github.com/cockroachdb/cockroach/pkg/sql/execinfra.(*ProcessorBase).DrainHelper
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/execinfra/processorsbase.go:642
*   | github.com/cockroachdb/cockroach/pkg/sql.(*planNodeToRowSource).Next
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/plan_node_to_row_source.go:183
*   | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*Columnarizer).Next
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/colexec/columnarizer.go:199
*   | github.com/cockroachdb/cockroach/pkg/sql/colflow.(*batchInfoCollector).Next
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/colflow/stats.go:91
*   | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*Materializer).next
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/colexec/materializer.go:273
*   | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*Materializer).nextAdapter
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/colexec/materializer.go:298
*   | github.com/cockroachdb/cockroach/pkg/sql/colexecerror.CatchVectorizedRuntimeError
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/colexecerror/error.go:91
*   | github.com/cockroachdb/cockroach/pkg/sql/colexec.(*Materializer).Next
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/colexec/materializer.go:304
*   | github.com/cockroachdb/cockroach/pkg/sql/execinfra.Run
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/execinfra/base.go:171
*   | github.com/cockroachdb/cockroach/pkg/sql/execinfra.(*ProcessorBase).Run
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/execinfra/processorsbase.go:774
*   | github.com/cockroachdb/cockroach/pkg/sql/flowinfra.(*FlowBase).Run
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/flowinfra/flow.go:382
*   | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).Run
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/distsql_running.go:359
*   | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).planAndRunPostquery
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/distsql_running.go:1156
*   | github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).PlanAndRunCascadesAndChecks
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/distsql_running.go:1096
*   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execWithDistSQLEngine
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/conn_executor_exec.go:1050
*   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).dispatchToExecutionEngine
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/conn_executor_exec.go:912
*   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/conn_executor_exec.go:656
*   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/conn_executor_exec.go:120
*   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd.func1
*   |   /home/andrei/src/github.com/cockroachdb/cockroach-2/pkg/sql/conn_executor.go:1514
* Wraps: (4) runtime error: invalid memory address or nil pointer dereference

[yuzefovich]

@andreimatei what SHA was it on for master? I think it should be fixed by #62126 which was merged yesterday.

[yuzefovich]

Well, I'm actually not sure whether this issue is a dup of #62061 cause it doesn't look like an error occurred in Init. Thanks for the report!

[andreimatei]

Still happens after #62126

[asubiotto]

Heads up that this is one of two issues blocking the release team from picking a beta sha tomorrow.

[yuzefovich]

I'm not sure whether it qualifies as a beta release blocker, but the impact is that running EXPLAIN ANALYZE (DEBUG) via the vectorized engine on the queries that contain unsupported by DistSQL planNodes (for example, all mutations) will result in a panic. It seems pretty bad for me, and given that the fix is out, I'll keep the label on.

[yuzefovich]

62409 reverted the introduction of EXPLAIN (VEC) files to the stmt bundle on release-21.1 branch, so I'm removing the release blocker labels from this issue.