search

pingcap / tidb

TiDB - the open-source, cloud-native, distributed SQL database designed for modern applications.
https://pingcap.com
Apache License 2.0
37.28k stars 5.84k forks source link

runtime error: slice bounds out of range [:33] with capacity 32 in `expression.(*builtinRpadUTF8Sig).vecEvalString` #53797

Open r33s3n6 opened 5 months ago

r33s3n6 commented 5 months ago

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

create table t_u6z6v6m3u ( 
c_ytzsa int ,
c_zllrsddn96 text ,
c_gvxf double ,
c_g91_kal4n double unique ,
primary key(c_ytzsa) CLUSTERED) pre_split_regions=2;

insert into t_u6z6v6m3u (c_ytzsa, c_zllrsddn96, c_gvxf, c_g91_kal4n) values 
  (666539487, 'w5qtrc', 85.90, 32768.9), 
  (1408667566, 'a6d0', 71.4, 15.89), 
  (-1680147087, 'yit7oneb5', 65535.7, 257.3), 
  (1211374988, 'e64', 92.46, 11.76);

SELECT
  rpad(
      cast(ref_0.c_zllrsddn96 as char), 
      33, 
      cast(unhex(
        cast(case when (NOT NOT(cast( (cast(null as char) >= 'g7ru0xq_') as unsigned))) then ref_0.c_zllrsddn96 else ref_0.c_zllrsddn96 end
           as char)) as char)) as c4
FROM
  t_u6z6v6m3u as ref_0;

2. What did you expect to see? (Required)

Expect no crashes

3. What did you see instead (Required)

runtime error: slice bounds out of range [:33] with capacity 32
github.com/pingcap/errors.AddStack
    /root/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20240318064555-6bd07397691f/errors.go:178
github.com/pingcap/errors.Trace
    /root/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20240318064555-6bd07397691f/juju_adaptor.go:15
github.com/pingcap/tidb/pkg/util.GetRecoverError
    /workspace/source/tidb/pkg/util/util.go:304
github.com/pingcap/tidb/pkg/executor.recoveryProjection
    /workspace/source/tidb/pkg/executor/projection.go:466
github.com/pingcap/tidb/pkg/executor.(*projectionWorker).run.func1
    /workspace/source/tidb/pkg/executor/projection.go:435
runtime.gopanic
    /usr/local/go/src/runtime/panic.go:914
runtime.goPanicSliceAcap
    /usr/local/go/src/runtime/panic.go:140
github.com/pingcap/tidb/pkg/expression.(*builtinRpadUTF8Sig).vecEvalString
    /workspace/source/tidb/pkg/expression/builtin_string_vec.go:2630
github.com/pingcap/tidb/pkg/expression.(*ScalarFunction).VecEvalString
    /workspace/source/tidb/pkg/expression/scalar_function.go:72
github.com/pingcap/tidb/pkg/expression.evalOneVec
    /workspace/source/tidb/pkg/expression/chunk_executor.go:172
github.com/pingcap/tidb/pkg/expression.(*defaultEvaluator).run
    /workspace/source/tidb/pkg/expression/evaluator.go:52
github.com/pingcap/tidb/pkg/expression.(*EvaluatorSuite).Run
    /workspace/source/tidb/pkg/expression/evaluator.go:124
github.com/pingcap/tidb/pkg/executor.(*projectionWorker).run
    /workspace/source/tidb/pkg/executor/projection.go:451
runtime.goexit
    /usr/local/go/src/runtime/asm_amd64.s:1650

4. What is your TiDB version? (Required)

Release Version: v8.1.0
Edition: Community
Git Commit Hash: 945d07c5d5c7a1ae212f6013adfb187f2de24b23
Git Branch: HEAD
UTC Build Time: 2024-05-21 03:51:57
GoVersion: go1.21.10
Race Enabled: false
Check Table Before Drop: false
Store: tikv

We are the BASS team from the School of Cyber Science and Technology at Beihang University. Our main focus is on system software security, operating systems, and program analysis research, as well as the development of automated program testing frameworks for detecting software defects. Using our self-developed database vulnerability testing tool, we have identified the above-mentioned vulnerabilities in TiDB that may lead to database crashes.

Defined2014 commented 5 months ago

Can't reproduce it, it will report error like

mysql> SELECT
    ->   rpad(
    ->       cast(ref_0.c_zllrsddn96 as char),
    ->       33,
    ->       cast(unhex(
    ->         cast(case when (NOT NOT(cast( (cast(null as char) >= 'g7ru0xq_') as unsigned))) then ref_0.c_zllrsddn96 else ref_0.c_zllrsddn96 end
    ->            as char)) as char)) as c4
    -> FROM
    ->   t_u6z6v6m3u as ref_0;
ERROR 3854 (HY000): Cannot convert string '\xA6\xD0' from binary to utf8
r33s3n6 commented 5 months ago

Seems that when we are using MySQL ODBC connector, the error can be triggered:

[2024/06/07 05:14:30.316 +00:00] [ERROR] [projection.go:468] ["projection executor panicked"] [error="runtime error: slice bounds out of range [:33] with capacity 32"] [stack="github.com/pingcap/tidb/pkg/executor.recoveryProjection\n\t/workspace/source/tidb/pkg/executor/projection.go:468\ngithub.com/pingcap/tidb/pkg/executor.(*projectionWorker).run.func1\n\t/workspace/source/tidb/pkg/executor/projection.go:435\nruntime.gopanic\n\t/usr/local/go/src/runtime/panic.go:914\nruntime.goPanicSliceAcap\n\t/usr/local/go/src/runtime/panic.go:140\ngithub.com/pingcap/tidb/pkg/expression.(*builtinRpadUTF8Sig).vecEvalString\n\t/workspace/source/tidb/pkg/expression/builtin_string_vec.go:2630\ngithub.com/pingcap/tidb/pkg/expression.(*ScalarFunction).VecEvalString\n\t/workspace/source/tidb/pkg/expression/scalar_function.go:72\ngithub.com/pingcap/tidb/pkg/expression.evalOneVec\n\t/workspace/source/tidb/pkg/expression/chunk_executor.go:172\ngithub.com/pingcap/tidb/pkg/expression.(*defaultEvaluator).run\n\t/workspace/source/tidb/pkg/expression/evaluator.go:52\ngithub.com/pingcap/tidb/pkg/expression.(*EvaluatorSuite).Run\n\t/workspace/source/tidb/pkg/expression/evaluator.go:124\ngithub.com/pingcap/tidb/pkg/executor.(*projectionWorker).run\n\t/workspace/source/tidb/pkg/executor/projection.go:451"]
[2024/06/07 05:14:30.318 +00:00] [INFO] [conn.go:1151] ["command dispatched failed"] [conn=1776304932] [session_alias=] [connInfo="id:1776304932, addr:10.1.2.1:50598 status:10, collation:utf8mb4_0900_ai_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="SELECT   rpad(       cast(ref_0.c_zllrsddn96 as char),        33,        cast(unhex(         cast(case when (NOT NOT(cast( (cast(null as char) >= 'g7ru0xq_') as unsigned))) then ref_0.c_zllrsddn96 else ref_0.c_zllrsddn96 end            as char)) as char)) as c4 FROM   t_u6z6v6m3u as ref_0;"] [txn_mode=PESSIMISTIC] [timestamp=450294518978707470] [err="runtime error: slice bounds out of range [:33] with capacity 32\ngithub.com/pingcap/errors.AddStack\n\t/root/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20240318064555-6bd07397691f/errors.go:178\ngithub.com/pingcap/errors.Trace\n\t/root/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20240318064555-6bd07397691f/juju_adaptor.go:15\ngithub.com/pingcap/tidb/pkg/util.GetRecoverError\n\t/workspace/source/tidb/pkg/util/util.go:304\ngithub.com/pingcap/tidb/pkg/executor.recoveryProjection\n\t/workspace/source/tidb/pkg/executor/projection.go:466\ngithub.com/pingcap/tidb/pkg/executor.(*projectionWorker).run.func1\n\t/workspace/source/tidb/pkg/executor/projection.go:435\nruntime.gopanic\n\t/usr/local/go/src/runtime/panic.go:914\nruntime.goPanicSliceAcap\n\t/usr/local/go/src/runtime/panic.go:140\ngithub.com/pingcap/tidb/pkg/expression.(*builtinRpadUTF8Sig).vecEvalString\n\t/workspace/source/tidb/pkg/expression/builtin_string_vec.go:2630\ngithub.com/pingcap/tidb/pkg/expression.(*ScalarFunction).VecEvalString\n\t/workspace/source/tidb/pkg/expression/scalar_function.go:72\ngithub.com/pingcap/tidb/pkg/expression.evalOneVec\n\t/workspace/source/tidb/pkg/expression/chunk_executor.go:172\ngithub.com/pingcap/tidb/pkg/expression.(*defaultEvaluator).run\n\t/workspace/source/tidb/pkg/expression/evaluator.go:52\ngithub.com/pingcap/tidb/pkg/expression.(*EvaluatorSuite).Run\n\t/workspace/source/tidb/pkg/expression/evaluator.go:124\ngithub.com/pingcap/tidb/pkg/executor.(*projectionWorker).run\n\t/workspace/source/tidb/pkg/executor/projection.go:451\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1650"]

But when we are using MySQL shell, it just shows Cannot convert string '\xA6\xD0' from binary to utf8: 

[2024/06/07 05:13:29.533 +00:00] [INFO] [conn.go:1151] ["command dispatched failed"] [conn=1776304908] [session_alias=] [connInfo="id:1776304908, addr:10.1.2.1:52446 status:10, collation:utf8mb4_0900_ai_ci, user:root"] [command=Query] [status="inTxn:0, autocommit:1"] [sql="SELECT   rpad(       cast(ref_0.c_zllrsddn96 as char),        33,        cast(unhex(         cast(case when (NOT NOT(cast( (cast(null as char) >= 'g7ru0xq_') as unsigned))) then ref_0.c_zllrsddn96 else ref_0.c_zllrsddn96 end            as char)) as char)) as c4 FROM   t_u6z6v6m3u as ref_0"] [txn_mode=PESSIMISTIC] [timestamp=450294503053459457] [err="[expression:3854]Cannot convert string '\\xA6\\xD0' from binary to utf8mb4\ngithub.com/pingcap/errors.AddStack\n\t/root/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20240318064555-6bd07397691f/errors.go:178\ngithub.com/pingcap/errors.(*Error).GenWithStackByArgs\n\t/root/go/pkg/mod/github.com/pingcap/errors@v0.11.5-0.20240318064555-6bd07397691f/normalize.go:175\ngithub.com/pingcap/tidb/pkg/expression.(*builtinInternalFromBinarySig).vecEvalString\n\t/workspace/source/tidb/pkg/expression/builtin_convert_charset.go:228\ngithub.com/pingcap/tidb/pkg/expression.(*ScalarFunction).VecEvalString\n\t/workspace/source/tidb/pkg/expression/scalar_function.go:72\ngithub.com/pingcap/tidb/pkg/expression.(*builtinCastStringAsStringSig).vecEvalString\n\t/workspace/source/tidb/pkg/expression/builtin_cast_vec.go:1919\ngithub.com/pingcap/tidb/pkg/expression.(*ScalarFunction).VecEvalString\n\t/workspace/source/tidb/pkg/expression/scalar_function.go:72\ngithub.com/pingcap/tidb/pkg/expression.(*builtinRpadUTF8Sig).vecEvalString\n\t/workspace/source/tidb/pkg/expression/builtin_string_vec.go:2593\ngithub.com/pingcap/tidb/pkg/expression.(*ScalarFunction).VecEvalString\n\t/workspace/source/tidb/pkg/expression/scalar_function.go:72\ngithub.com/pingcap/tidb/pkg/expression.evalOneVec\n\t/workspace/source/tidb/pkg/expression/chunk_executor.go:172\ngithub.com/pingcap/tidb/pkg/expression.(*defaultEvaluator).run\n\t/workspace/source/tidb/pkg/expression/evaluator.go:52\ngithub.com/pingcap/tidb/pkg/expression.(*EvaluatorSuite).Run\n\t/workspace/source/tidb/pkg/expression/evaluator.go:124\ngithub.com/pingcap/tidb/pkg/executor.(*projectionWorker).run\n\t/workspace/source/tidb/pkg/executor/projection.go:451\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1650"]

There are slight differences in stack trace, the first one:

...
github.com/pingcap/tidb/pkg/expression.(*builtinRpadUTF8Sig).vecEvalString
    /workspace/source/tidb/pkg/expression/builtin_string_vec.go:2630
...

the second one:

...
github.com/pingcap/tidb/pkg/expression.(*ScalarFunction).VecEvalString
    /workspace/source/tidb/pkg/expression/scalar_function.go:72
github.com/pingcap/tidb/pkg/expression.(*builtinRpadUTF8Sig).vecEvalString
    /workspace/source/tidb/pkg/expression/builtin_string_vec.go:2593
...
Release Version: v8.2.0-alpha-216-gfe5858b
Edition: Community
Git Commit Hash: fe5858b00cd63808ac414c6e102a353778b0aaa7
Git Branch: HEAD
UTC Build Time: 2024-05-23 01:44:42
GoVersion: go1.21.10
Race Enabled: false
Check Table Before Drop: false
Store: tikv
bb7133 commented 4 hours ago

Change the tag to 'sig-execution' since it is related to expression evaluation.