in(-1,map_values(null)) only common path throw

[kagamiori]

There are two ways to reproduce this error.

1. Download the attached files of the saved input and expression. Then run velox/expression/tests/velox_expression_runner_test --input_path "/home/weihe/fuzzer_repro/velox_vector_xWVO02" --sql_path "/home/weihe/fuzzer_repro/velox_sql_1KLk7v" --result_path "/home/weihe/fuzzer_repro/velox_vector_yChHFS" (change the file paths to yours).
2. (internal only) Alternatively, check out 078ffeddc and then run velox/expression/tests/velox_expression_fuzzer_test --velox_fuzzer_enable_complex_types --velox_fuzzer_max_level_of_nesting 2 --only "array_position,in,map_values" --seed 2768396557.

======> Started iteration 0 (seed: 2768396557)
I1026 14:25:40.879932 3101312 ExpressionVerifier.cpp:130] Executing expression: in(-1,map_values(null))
I1026 14:25:40.880069 3101312 ExpressionVerifier.cpp:133] 0 vectors as input:
E1026 14:25:40.882357 3101312 Exceptions.h:68] Line: velox/functions/prestosql/InPredicate.cpp:37, Function:toValues, Expression: size > 0 (0 vs. 0) IN list must not be empty, Source: USER, ErrorCode: INVALID_ARGUMENT
E1026 14:25:40.883929 3101312 ExpressionVerifier.cpp:67] Only common path threw exception:
I1026 14:25:40.884773 3101312 ExpressionVerifier.cpp:288] Persisted input at '/home/weihe/fuzzer_repro/velox_vector_xWVO02' and result at '/home/weihe/fuzzer_repro/velox_vector_yChHFS' and sql at '/home/weihe/fuzzer_repro/velox_sql_1KLk7v'
terminate called after throwing an instance of 'facebook::velox::VeloxUserError'
  what():  Exception: VeloxUserError
Error Source: USER
Error Code: INVALID_ARGUMENT
Reason: (0 vs. 0) IN list must not be empty
Retriable: False
Expression: size > 0
Function: toValues
File: velox/functions/prestosql/InPredicate.cpp
Line: 37
Stack trace:
Stack trace has been disabled.Use --velox_exception_user_stacktrace=true to enable it.

*** Aborted at 1666819540 (Unix time, try 'date -d @1666819540') ***
*** Signal 6 (SIGABRT) (0x37b3c002f5280) received by PID 3101312 (pthread TID 0x7f726c78fec0) (linux TID 3101312) (maybe from PID 3101312, UID 228156) (code: -6), stack trace: ***
    @ 0000000000010fee folly::symbolizer::(anonymous namespace)::innerSignalHandler(int, siginfo_t*, void*)
                       ./folly/experimental/symbolizer/SignalHandler.cpp:449
    @ 000000000000f711 folly::symbolizer::(anonymous namespace)::signalHandler(int, siginfo_t*, void*)
                       ./folly/experimental/symbolizer/SignalHandler.cpp:470
    @ 0000000000000000 (unknown)
    @ 000000000009c9d3 __GI___pthread_kill
    @ 00000000000444ec __GI_raise
    @ 000000000002c432 __GI_abort
    @ 00000000000a3fd4 __gnu_cxx::__verbose_terminate_handler()
    @ 00000000000a1b39 __cxxabiv1::__terminate(void (*)())
    @ 00000000000a1ba4 std::terminate()
    @ 00000000000a1ec1 __cxa_rethrow
    @ 000000000001ec9c facebook::velox::test::ExpressionVerifier::verify(std::shared_ptr<facebook::velox::core::ITypedExpr const> const&, std::shared_ptr<facebook::velox::RowVector> const&, std::shared_ptr<facebook::velox::BaseVector>&&, bool)
                       ./velox/expression/tests/ExpressionVerifier.cpp:227
    @ 0000000000095986 facebook::velox::test::ExpressionFuzzer::go()
                       ./velox/expression/tests/ExpressionFuzzer.cpp:727
    @ 00000000000d8cfd facebook::velox::test::expressionFuzzer(std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::vector<facebook::velox::exec::FunctionSignature const*, std::allocator<facebook::velox::exec::FunctionSignature const*> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::vector<facebook::velox::exec::FunctionSignature const*, std::allocator<facebook::velox::exec::FunctionSignature const*> > > > >, unsigned long)
                       ./velox/expression/tests/ExpressionFuzzer.cpp:754
    @ 00000000002c30e3 FuzzerRunner::run(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long, std::unordered_set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)
                       ./velox/expression/tests/FuzzerRunner.h:108
                       -> ./velox/expression/tests/ExpressionFuzzerTest.cpp
    @ 00000000002c29ac main
                       ./velox/expression/tests/ExpressionFuzzerTest.cpp:57
    @ 000000000002c656 __libc_start_call_main
    @ 000000000002c717 __libc_start_main_alias_2
    @ 00000000002b8b80 _start
                       /home/engshare/third-party2/glibc/2.34/src/glibc-2.34/csu/../sysdeps/x86_64/start.S:116
Aborted (core dumped)

[kagamiori]

in_map_values_repro.zip

[mbasmanova]

Here, common path does constant folding turning map_values(null) into a null constant. It then attempts to create an instance of a stateful vector function InPredicate with a constant null second argument. The creation throws. Common path doesn't do constant folding and creates an instance of InPredicate with no constant inputs. The creation succeeds. Common path then evaluates inputs to "in" function, eliminates rows where at least one of the inputs is null (in this case all rows are eliminated because second argument is null) and return nulls without calling InPredicate.

First, failures during creation of InPredicate need to be suppressed and re-thrown (via context.setErrors(rows, exception)) if and when InPredicate::apply is called.

Second, constant folding should be more powerful and apply to function calls with constant null inputs. If these have default null behavior they can be constant folded into a null.

CC: @laithsakka @kagamiori @bikramSingh91

[mbasmanova]

@oerling Orri offered to help with this one.

[mbasmanova]

Part of the fix: #3050

[kagamiori]

Second, constant folding should be more powerful and apply to function calls with constant null inputs. If these have default null behavior they can be constant folded into a null.

I had a PR for the second part back in August: https://github.com/facebookincubator/velox/pull/2007. It was blocked by fuzzer test failures that have the same root cause as https://github.com/facebookincubator/velox/issues/2749 which has been fixed by https://github.com/facebookincubator/velox/pull/2961 recently.

[kagamiori]

Second, constant folding should be more powerful and apply to function calls with constant null inputs. If these have default null behavior they can be constant folded into a null.

I had a PR for the second part back in August: #2007. It was blocked by fuzzer test failures that have the same root cause as #2749 which has been fixed by #2961 recently.

OK, the PR #2007 is blocked by another fuzzer test failure again (https://github.com/facebookincubator/velox/issues/3060). The failure can be reproduced via the following command.

buck-out/gen/velox/expression/tests/velox_expression_fuzzer_test --only "date_format,from_unixtime,replace,chr,substr,regexp_replace,regexp_extract,split_part,url_extract_path,lpad,strrpos,strpos,bitwise_and,truncate,rpad,to_base64,sha512" --seed 2869762080

[mbasmanova]

Possible fix from @oerling #3189