search

facebookincubator / velox

A C++ vectorized database acceleration library aimed to optimizing query engines and data processing systems.
https://velox-lib.io/
Apache License 2.0
3.45k stars 1.13k forks source link

Unicode characters are not supported for conversion to integer types #10871

Open wypb opened 1 month ago

wypb commented 1 month ago

Bug description

After https://github.com/facebookincubator/velox/pull/10804, Presto Fuzzer CI will failed. The exception information is as follows: See this CI link

E20240827 04:34:58.161984    70 Exceptions.h:67] Line: /__w/velox/velox/velox/./velox/expression/CastExpr-inl.h:312, Function:applyCastKernel, Expression: functions::stringCore::isAscii( inputRowValue.data(), inputRowValue.size()) Unicode characters are not supported for conversion to integer types, Source: RUNTIME, ErrorCode: INVALID_STATE
E20240827 04:34:58.162415    70 ExpressionVerifier.cpp:146] Common eval: Exceptions other than VeloxUserError are not allowed.
terminate called after throwing an instance of 'facebook::velox::VeloxRuntimeError'
  what():  Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: Unicode characters are not supported for conversion to integer types
Retriable: False
Expression: functions::stringCore::isAscii( inputRowValue.data(), inputRowValue.size())
Context: cast((B   �+M*m�i�����:VARCHAR) as SMALLINT)
Function: applyCastKernel
File: /__w/velox/velox/velox/./velox/expression/CastExpr-inl.h
Line: 312
Stack trace:
# 0  _ZN8facebook5velox7process10StackTraceC1Ei
# 1  _ZN8facebook5velox14VeloxException5State4makeIZNS1_C4EPKcmS5_St17basic_string_viewIcSt11char_traitsIcEES9_S9_S9_bNS1_4TypeES9_EUlRT_E_EESt10shared_ptrIKS2_ESA_SB_
# 2  _ZN8facebook5velox14VeloxExceptionC1EPKcmS3_St17basic_string_viewIcSt11char_traitsIcEES7_S7_S7_bNS1_4TypeES7_
# 3  _ZN8facebook5velox17VeloxRuntimeErrorC2EPKcmS3_St17basic_string_viewIcSt11char_traitsIcEES7_S7_S7_bS7_
# 4  _ZN8facebook5velox6detail14veloxCheckFailINS0_17VeloxRuntimeErrorEPKcEEvRKNS1_18VeloxCheckFailArgsET0_
# 5  _ZN8facebook5velox4exec8CastExpr15applyCastKernelILNS0_8TypeKindE2ELS4_7ENS0_4util16PrestoCastPolicyEEEviRNS1_7EvalCtxEPKNS0_12SimpleVectorINS0_10TypeTraitsIXT0_EE10NativeTypeEEEPNS0_10FlatVectorINSA_IXT_EE10NativeTypeEEE
# 6  _ZZN8facebook5velox4exec8CastExpr19applyCastPrimitivesILNS0_8TypeKindE2ELS4_7EEEvRKNS0_17SelectivityVectorERNS1_7EvalCtxERKNS0_10BaseVectorERSt10shared_ptrISA_EENKUliE0_clEi
# 7  _ZNK8facebook5velox17SelectivityVector15applyToSelectedIZNS0_4exec8CastExpr27applyToSelectedNoThrowLocalIZNS4_19applyCastPrimitivesILNS0_8TypeKindE2ELS7_7EEEvRKS1_RNS3_7EvalCtxERKNS0_10BaseVectorERSt10shared_ptrISC_EEUliE0_EEvSB_S9_SH_OT_EUlSJ_E0_EEvSJ_
# 8  _ZN8facebook5velox4exec8CastExpr27applyToSelectedNoThrowLocalIZNS2_19applyCastPrimitivesILNS0_8TypeKindE2ELS5_7EEEvRKNS0_17SelectivityVectorERNS1_7EvalCtxERKNS0_10BaseVectorERSt10shared_ptrISB_EEUliE0_EEvSA_S8_SG_OT_
# 9  _ZN8facebook5velox4exec8CastExpr19applyCastPrimitivesILNS0_8TypeKindE2ELS4_7EEEvRKNS0_17SelectivityVectorERNS1_7EvalCtxERKNS0_10BaseVectorERSt10shared_ptrISA_E
# 10 _ZZN8facebook5velox4exec8CastExpr27applyCastPrimitivesDispatchILNS0_8TypeKindE2EEEvRKSt10shared_ptrIKNS0_4TypeEESA_RKNS0_17SelectivityVectorERNS1_7EvalCtxERKNS0_10BaseVectorERS5_ISG_EENKUlvE_clEv
# 11 _ZN8facebook5velox4exec8CastExpr27applyCastPrimitivesDispatchILNS0_8TypeKindE2EEEvRKSt10shared_ptrIKNS0_4TypeEESA_RKNS0_17SelectivityVectorERNS1_7EvalCtxERKNS0_10BaseVectorERS5_ISG_E
# 12 _ZZN8facebook5velox4exec8CastExpr11applyPeeledERKNS0_17SelectivityVectorERKNS0_10BaseVectorERNS1_7EvalCtxERKSt10shared_ptrIKNS0_4TypeEESG_RSB_IS6_EENKUlvE3_clEv
# 13 _ZN8facebook5velox4exec8CastExpr11applyPeeledERKNS0_17SelectivityVectorERKNS0_10BaseVectorERNS1_7EvalCtxERKSt10shared_ptrIKNS0_4TypeEESG_RSB_IS6_E
# 14 _ZZN8facebook5velox4exec8CastExpr5applyERKNS0_17SelectivityVectorERKSt10shared_ptrINS0_10BaseVectorEERNS1_7EvalCtxERKS6_IKNS0_4TypeEESH_RS8_ENKUlRNS1_12ContextSaverEE_clESK_
# 15 _ZN8facebook5velox4exec16withContextSaverIZNS1_8CastExpr5applyERKNS0_17SelectivityVectorERKSt10shared_ptrINS0_10BaseVectorEERNS1_7EvalCtxERKS7_IKNS0_4TypeEESI_RS9_EUlRNS1_12ContextSaverEE_EEvOT_
# 16 _ZN8facebook5velox4exec8CastExpr5applyERKNS0_17SelectivityVectorERKSt10shared_ptrINS0_10BaseVectorEERNS1_7EvalCtxERKS6_IKNS0_4TypeEESH_RS8_
# 17 _ZN8facebook5velox4exec8CastExpr15evalSpecialFormERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 18 _ZN8facebook5velox4exec4Expr24evalSpecialFormWithStatsERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 19 _ZN8facebook5velox4exec4Expr11evalAllImplERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 20 _ZN8facebook5velox4exec4Expr7evalAllERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 21 _ZN8facebook5velox4exec4Expr13evalWithNullsERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 22 _ZN8facebook5velox4exec4Expr13evalEncodingsERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEE
# 23 _ZN8facebook5velox4exec4Expr4evalERKNS0_17SelectivityVectorERNS1_7EvalCtxERSt10shared_ptrINS0_10BaseVectorEEPKNS1_7ExprSetE
# 24 _ZN8facebook5velox4exec12_GLOBAL__N_117tryFoldIfConstantERKSt10shared_ptrINS1_4ExprEEPNS2_5ScopeE
# 25 _ZN8facebook5velox4exec12_GLOBAL__N_1[26](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:27)compileRewrittenExpressionERKSt10shared_ptrIKNS0_4core10ITypedExprEEPNS2_5ScopeERKNS4_11QueryConfigEPNS0_6memory10MemoryPoolERKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashISO_ESt8equal_toISO_ESaISO_EEb
# 26 _ZN8facebook5velox4exec12_GLOBAL__N_117compileExpressionERKSt10shared_ptrIKNS0_4core10ITypedExprEEPNS2_5ScopeERKNS4_11QueryConfigEPNS0_6memory10MemoryPoolERKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashISO_ESt8equal_toISO_ESaISO_EEb
# [27](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:28) _ZN8facebook5velox4exec12_GLOBAL__N_113compileInputsERKSt10shared_ptrIKNS0_4core10ITypedExprEEPNS2_5ScopeERKNS4_11QueryConfigEPNS0_6memory10MemoryPoolERKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashISO_ESt8equal_toISO_ESaISO_EEb
# [28](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:29) _ZN8facebook5velox4exec12_GLOBAL__N_126compileRewrittenExpressionERKSt10shared_ptrIKNS0_4core10ITypedExprEEPNS2_5ScopeERKNS4_11QueryConfigEPNS0_6memory10MemoryPoolERKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashISO_ESt8equal_toISO_ESaISO_EEb
# [29](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:30) _ZN8facebook5velox4exec12_GLOBAL__N_117compileExpressionERKSt10shared_ptrIKNS0_4core10ITypedExprEEPNS2_5ScopeERKNS4_11QueryConfigEPNS0_6memory10MemoryPoolERKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashISO_ESt8equal_toISO_ESaISO_EEb
# [30](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:31) _ZN8facebook5velox4exec12_GLOBAL__N_113compileInputsERKSt10shared_ptrIKNS0_4core10ITypedExprEEPNS2_5ScopeERKNS4_11QueryConfigEPNS0_6memory10MemoryPoolERKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashISO_ESt8equal_toISO_ESaISO_EEb
# [31](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:32) _ZN8facebook5velox4exec12_GLOBAL__N_126compileRewrittenExpressionERKSt10shared_ptrIKNS0_4core10ITypedExprEEPNS2_5ScopeERKNS4_11QueryConfigEPNS0_6memory10MemoryPoolERKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashISO_ESt8equal_toISO_ESaISO_EEb
# [32](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:33) _ZN8facebook5velox4exec12_GLOBAL__N_117compileExpressionERKSt10shared_ptrIKNS0_4core10ITypedExprEEPNS2_5ScopeERKNS4_11QueryConfigEPNS0_6memory10MemoryPoolERKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashISO_ESt8equal_toISO_ESaISO_EEb
# [33](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:34) _ZN8facebook5velox4exec12_GLOBAL__N_113compileInputsERKSt10shared_ptrIKNS0_4core10ITypedExprEEPNS2_5ScopeERKNS4_11QueryConfigEPNS0_6memory10MemoryPoolERKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashISO_ESt8equal_toISO_ESaISO_EEb
# [34](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:35) _ZN8facebook5velox4exec12_GLOBAL__N_126compileRewrittenExpressionERKSt10shared_ptrIKNS0_4core10ITypedExprEEPNS2_5ScopeERKNS4_11QueryConfigEPNS0_6memory10MemoryPoolERKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashISO_ESt8equal_toISO_ESaISO_EEb
# [35](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:36) _ZN8facebook5velox4exec12_GLOBAL__N_117compileExpressionERKSt10shared_ptrIKNS0_4core10ITypedExprEEPNS2_5ScopeERKNS4_11QueryConfigEPNS0_6memory10MemoryPoolERKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashISO_ESt8equal_toISO_ESaISO_EEb
# [36](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:37) _ZN8facebook5velox4exec12_GLOBAL__N_113compileInputsERKSt10shared_ptrIKNS0_4core10ITypedExprEEPNS2_5ScopeERKNS4_11QueryConfigEPNS0_6memory10MemoryPoolERKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashISO_ESt8equal_toISO_ESaISO_EEb
# [37](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:38) _ZN8facebook5velox4exec12_GLOBAL__N_126compileRewrittenExpressionERKSt10shared_ptrIKNS0_4core10ITypedExprEEPNS2_5ScopeERKNS4_11QueryConfigEPNS0_6memory10MemoryPoolERKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashISO_ESt8equal_toISO_ESaISO_EEb
# [38](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:39) _ZN8facebook5velox4exec12_GLOBAL__N_117compileExpressionERKSt10shared_ptrIKNS0_4core10ITypedExprEEPNS2_5ScopeERKNS4_11QueryConfigEPNS0_6memory10MemoryPoolERKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashISO_ESt8equal_toISO_ESaISO_EEb
# [39](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:40) _ZN8facebook5velox4exec18compileExpressionsERKSt6vectorISt10shared_ptrIKNS0_4core10ITypedExprEESaIS7_EEPNS4_7ExecCtxEPNS1_7ExprSetEb
# [40](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:41) _ZN8facebook5velox4exec7ExprSetC2ERKSt6vectorISt10shared_ptrIKNS0_4core10ITypedExprEESaIS8_EEPNS5_7ExecCtxEb
# 41 _ZN8facebook5velox4test18ExpressionVerifier6verifyERKSt6vectorISt10shared_ptrIKNS0_4core10ITypedExprEESaIS8_EERKS4_INS0_9RowVectorEEOS4_INS0_10BaseVectorEEbS3_IiSaIiEE
# [42](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:43) _ZN8facebook5velox6fuzzer24ExpressionFuzzerVerifier2goEv
# 43 _ZN8facebook5velox6fuzzer12FuzzerRunner12runFromGtestEmRKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashIS9_ESt8equal_toIS9_ESaIS9_EERKSt13unordered_mapIS9_S9_SB_SD_SaISt4pairIKS9_S9_EEE
# [44](https://github.com/facebookincubator/velox/actions/runs/10571351442/job/29288175420?pr=10372#step:6:45) _ZN8facebook5velox6fuzzer12FuzzerRunner3runEmRKSt13unordered_setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4hashIS9_ESt8equal_toIS9_ESaIS9_EERKSt13unordered_mapIS9_S9_SB_SD_SaISt4pairIKS9_S9_EEE
# 45 main
# 46 __libc_start_call_main
# 47 __libc_start_main
# 48 _start

System information

Velox System Info v0.0.2 Commit: ba8c9517656d9a7907d89806018d6f9149d3bb80 CMake Version: 3.28.3 System: Linux-5.4.241-1-tlinux4-0017.7 Arch: x86_64 C++ Compiler: /usr/bin/c++ C++ Compiler Version: 12.2.1 C Compiler: /usr/bin/cc C Compiler Version: 12.2.1 CMake Prefix Path: /usr/local;/usr;/;/usr/local/lib64/python3.9/site-packages/cmake/data;/usr/local;/usr/X11R6;/usr/pkg;/opt

Relevant logs

No response

wypb commented 1 month ago

CC: @kgpai @kagamiori

kgpai commented 1 month ago

@wypb Thank you for brining this to our attention.

I am working on a fuzzer fix. We recently added a change that causes cast from varchar to integral types to throw non user exception as its currently unsupported in Velox. Throwing a user error can unfortunately lead to correctness bugs if wrapped in a try , thus the need for a system bug which is treated as an error by Fuzzer.