search

yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
8.97k stars 1.07k forks source link

[DocDB][LST] ERROR: Schema packing not found: 57, available_versions: [56, 54, 52, 50, 48, 46, 44, 42, 40, 38, 36, 34, 32,......] #23427

Open shishir2001-yb opened 2 months ago

shishir2001-yb commented 2 months ago

Jira Link: DB-12348

Description

Version: 2.20.4.0-b50 LST and Universe logs: Added in Jira

$ cd ~/code/yugabyte-db
$ git checkout 1e6329f40e5c531c09ade7015278078682293ebd
$ ./yb_build.sh release
$ ./bin/yb-ctl --rf 3 create --tserver_flags "ysql_pg_conf_csv='yb_debug_log_docdb_requests=true'",yb_enable_read_committed_isolation=true,ysql_num_shards_per_tserver=1,ysql_yb_bnl_batch_size=1024,enable_wait_queues=true  --master_flags "ysql_pg_conf_csv='yb_debug_log_docdb_requests=true'",yb_enable_read_committed_isolation=true,ysql_num_shards_per_tserver=1,ysql_yb_bnl_batch_size=1024
$ cd ~/code/yb-long-system-test
$ git checkout 75b1683c-dirty
$ ./long_system_test.py --runtime=0 --threads=10 --seed=773252 --complexity=full --scenario=cancel_session

Query: SELECT c0_int % 70 FROM t2 WHERE c0_int < -51 ORDER BY 1 LIMIT 8;

Error: ERROR: Schema packing not found: 57, available_versions: [56, 54, 52, 50, 48, 46, 44, 42, 40, 38, 36, 34, 32, 30, 28, 26, 24, 22, 20, 18, 16, 14, 12, 10, 8, 6, 4, 2, 0, 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49, 51, 53, 55]

2024-08-07 16:01:34,978 worker_6   EXPLAIN (ANALYZE FALSE, VERBOSE TRUE, COSTS TRUE, BUFFERS FALSE, TIMING FALSE, SUMMARY FALSE, FORMAT JSON)
SELECT c3_int FROM t3 ORDER BY 1 OFFSET 6;
2024-08-07 16:01:34,979 worker_6   SELECT c3_int FROM t3 ORDER BY 1 OFFSET 6;
2024-08-07 16:01:34,979 worker_4   rollback
2024-08-07 16:01:34,980 worker_5   EXPLAIN (ANALYZE TRUE, VERBOSE FALSE, COSTS TRUE, BUFFERS FALSE, TIMING TRUE, SUMMARY FALSE, FORMAT XML)
SELECT c1_int + 98, c0_int FROM t3 WHERE c2_text = '63' LIMIT 8;
2024-08-07 16:01:34,980 worker_7   SET enable_parallel_append TO off;
2024-08-07 16:01:34,980 worker_7   commit
2024-08-07 16:01:34,981 worker_7   SELECT c0_text FROM t4 WHERE c0_text = '-25' LIMIT 9 OFFSET 19;
2024-08-07 16:01:34,981 worker_9   SELECT ascii(c0_text) FROM t4 ORDER BY 1 LIMIT 6;
2024-08-07 16:01:34,983 worker_5   SELECT c1_int + 98, c0_int FROM t3 WHERE c2_text = '63' LIMIT 8;
2024-08-07 16:01:34,983 worker_6   commit
2024-08-07 16:01:34,984 worker_6   SELECT c2_text, octet_length(c0_text) FROM t0 WHERE c2_text = '66' ORDER BY 1 LIMIT 7 OFFSET 1;
2024-08-07 16:01:34,984 worker_7   commit
2024-08-07 16:01:34,984 worker_0   commit
2024-08-07 16:01:34,985 worker_7   EXPLAIN (ANALYZE FALSE, VERBOSE TRUE, COSTS TRUE, BUFFERS FALSE, TIMING FALSE, SUMMARY TRUE, FORMAT YAML)
SELECT c0_int - -17 FROM t2 WHERE c0_int <= 6 OFFSET 9;
2024-08-07 16:01:34,985 worker_4   INSERT INTO t1 VALUES ('59', -50, -78, '-9');
2024-08-07 16:01:34,985 worker_5   commit
2024-08-07 16:01:34,985 worker_5   EXPLAIN (ANALYZE TRUE, VERBOSE FALSE, COSTS FALSE, BUFFERS TRUE, TIMING FALSE, SUMMARY TRUE, FORMAT YAML)
SELECT c1_text FROM t0 WHERE c2_text = '-27' ORDER BY 1 LIMIT 11 OFFSET 17;
2024-08-07 16:01:34,986 worker_7   SELECT c0_int - -17 FROM t2 WHERE c0_int <= 6 OFFSET 9;
2024-08-07 16:01:34,986 worker_3   Failed: Restart read required at: { read: { physical: 1723046494983547 } local_limit: { physical: 1723046494983547 } global_limit: <min> in_txn_limit: <max> serial_no: 0 }
2024-08-07 16:01:34,986 worker_0   DELETE FROM t0 WHERE c0_text = '-39';
2024-08-07 16:01:34,987 worker_2   CREATE INDEX idx_t4_c0_text ON t4(c0_text);
2024-08-07 16:01:34,987 worker_1   Failed: Schema packing not found: 57, available_versions: [56, 54, 52, 50, 48, 46, 44, 42, 40, 38, 36, 34, 32, 30, 28, 26, 24, 22, 20, 18, 16, 14, 12, 10, 8, 6, 4, 2, 0, 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49, 51, 53, 55]

Schema:

2024-08-07 16:00:49,837 MainThread CREATE TYPE e0 AS ENUM ('e0_v0', 'e0_v1', 'e0_v2', 'e0_v3', 'e0_v4');
2024-08-07 16:00:49,882 MainThread CREATE TYPE e1 AS ENUM ('e1_v0', 'e1_v1', 'e1_v2', 'e1_v3');
2024-08-07 16:00:49,900 MainThread CREATE TYPE e2 AS ENUM ('e2_v0', 'e2_v1', 'e2_v2');
2024-08-07 16:00:49,917 MainThread CREATE TYPE e3 AS ENUM ('e3_v0', 'e3_v1', 'e3_v2', 'e3_v3', 'e3_v4');
2024-08-07 16:00:49,935 MainThread CREATE TYPE e4 AS ENUM ('e4_v0', 'e4_v1', 'e4_v2', 'e4_v3');
2024-08-07 16:00:49,954 MainThread CREATE TABLE t0(c0_text TEXT NOT NULL,
  c1_text TEXT DEFAULT '62',
  c2_text TEXT);
2024-08-07 16:00:50,087 MainThread CREATE TABLE t1(c0_text TEXT DEFAULT '-50',
  c1_int INT NOT NULL,
  c2_int INT DEFAULT -77,
  c3_text TEXT DEFAULT '67')
SPLIT INTO 10 TABLETS;
2024-08-07 16:00:50,301 MainThread CREATE TABLE t2(c0_int INT NOT NULL DEFAULT 29)
SPLIT INTO 2 TABLETS;
2024-08-07 16:00:50,441 MainThread CREATE TABLE t3(c0_int INT NOT NULL,
  c1_int INT NOT NULL DEFAULT 2147483647,
  c2_text TEXT NOT NULL DEFAULT '-19',
  c3_int INT);
2024-08-07 16:00:50,588 MainThread CREATE TABLE t4(c0_text TEXT);

Issue Type

kind/bug

Warning: Please confirm that this issue does not contain any sensitive information

Huqicheng commented 2 months ago

We might hit schema packing not found error with concurrent CREATE INDEX and SELECT in the following scenario:

txn1: create packed row decoder with latest version 56
txn2: alter schema and latest version = 57
txn3: write data to table with version 57
txn1: decode the data written by txn3 and get the error

Currently in ybdb, visibility and restart read required check is done after the rows are decoded. But if we find a row with larger schema version, read must be started prior to the write time of this row. We can ignore such rows during read.

according to @spolitov For instance we could store current schema version in DocReadContext , and ignore all rows with schema version after this one.