yugabyte / yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
https://www.yugabyte.com
Other
8.64k stars 1.04k forks source link

[YSQL] Cap the prefetch size so that it does not exceed Int32. #22411

Open yugabyte-ci opened 1 month ago

yugabyte-ci commented 1 month ago

Jira Link: DB-11320

Repro:

create table large (t text);
for i in {0..42}; do bin/ysqlsh -c "insert into large2 select repeat('abcdefghij', 1024 * 1024) from generate_series(1, 20);"; done
select * from large;

crashes with

F0530 08:53:54.237200 1866919936 casts.cc:21] Bad narrow cast: 4414509210 > 4294967295

(if there are 820 rows, the same select crashes with

F0530 08:52:53.170039 1891250176 yb_rpc.cc:457] Failed to serialize failure: Invalid argument (yb/rpc/serialization.cc:75): Sending too long RPC message (4257222773 bytes)
andrei-mart commented 1 month ago

There are two separate problems.

  1. Unsafe math on size_t type, which size is implementation specific, 32 bits in this particular case. We can enforce 64-bit type.
  2. The "too long RPC message" should be a regular error, properly presented to the user. User can solve that problem in a number of ways:
    • Increase max message size using a gflag (may require DBA privileges and tserver restart)
    • Reduce fetch size
    • Rewrite the queries to fetch shorter row (e.g. exclude some columns).