search

duckdb / duckdb-fuzzer

Repository that contains automatic reports of issues found through fuzzing DuckDB
0 stars 0 forks source link

AddressSanitizer error in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:99 #3237

Closed fuzzerofducks closed 2 months ago

fuzzerofducks commented 3 months ago

Issue found by SQLSmith on git commit hash f5ab7 using seed 78729220.

To Reproduce

create table all_types as select * exclude(small_enum, medium_enum, large_enum) from test_all_types();
SELECT 11 AS c0 FROM (SELECT 44 AS c0 FROM main.all_types AS ref_0 INNER JOIN main.all_types AS ref_1 ON ((ref_1."varchar" ~~ ref_1."varchar")) , (SELECT ref_1.timestamp_tz AS c0, ref_1."varchar" AS c1, ref_1.array_of_structs AS c2, ref_2.int_array AS c3, ref_1.blob AS c4, ref_0."bit" AS c5, ref_1.struct_of_arrays AS c6 FROM main.all_types AS ref_2 WHERE EXISTS(SELECT ref_2.fixed_nested_varchar_array AS c0 FROM main.all_types AS ref_3 WHERE (ref_1."union" IS NOT NULL) LIMIT 141) LIMIT 53) AS subq_0 WHERE EXISTS(SELECT subq_0.c4 AS c0 FROM main.all_types AS ref_4 WHERE 1 LIMIT 87) LIMIT 118) AS subq_1 INNER JOIN (SELECT ref_5.dec38_10 AS c0, ref_5."timestamp" AS c1, ref_5."smallint" AS c2, ref_5.array_of_structs AS c3, ref_5.ubigint AS c4 FROM main.all_types AS ref_5 WHERE (ref_5.usmallint IS NOT NULL)) AS subq_2 ON (EXISTS(SELECT subq_1.c0 AS c0, 83 AS c1 FROM main.all_types AS ref_6 WHERE (ref_6."varchar" ~~ ref_6."varchar") LIMIT 99)) INNER JOIN main.all_types AS ref_7 ON (((ref_7."varchar" !~~* ref_7."varchar") AND (ref_7."varchar" ^@ ref_7."varchar"))) LIMIT 90

Error Message

AddressSanitizer error in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:99
Tmonster commented 2 months ago

I'vebeen looking into this one and I think it comes down to DELIM join behavior.

This sql is a repro

FROM main.my_table ref_0
  INNER JOIN
      main.my_table AS ref_1
      ON (
          SELECT
              ref_1.struct_of_arrays AS c6
          FROM main.my_table AS ref_2
          WHERE
              EXISTS(SELECT ref_2.fixed_nested_varchar_array WHERE ref_1."union")
      ) and ref_1."varchar" ~~ ref_1."varchar";

And the plan looks like this 

┌───────────────────────────┐
│         PROJECTION        │
│    ────────────────────   │
│        Expressions:       │
│           union           │
│ fixed_nested_varchar_array│
│          varchar          │
│      struct_of_arrays     │
│           union           │
│ fixed_nested_varchar_array│
│          varchar          │
│      struct_of_arrays     │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│           FILTER          │
│    ────────────────────   │
│        Expressions:       │
│ CAST(SUBQUERY AS BOOLEAN) │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│         DELIM_JOIN        │
│    ────────────────────   │
│     Join Type: SINGLE     │
│                           ├───────────────────────────────────────────┐
│        Conditions:        │                                           │
│    (delim_index IS NOT    │                                           │
│ DISTINCT FROM delim_index)│                                           │
└─────────────┬─────────────┘                                           │
┌─────────────┴─────────────┐                             ┌─────────────┴─────────────┐
│           FILTER          │                             │         PROJECTION        │
│    ────────────────────   │                             │    ────────────────────   │
│        Expressions:       │                             │        Expressions:       │
│    (varchar ~~ varchar)   │                             │             c6            │
│                           │                             │        delim_index        │
└─────────────┬─────────────┘                             └─────────────┬─────────────┘
┌─────────────┴─────────────┐                             ┌─────────────┴─────────────┐
│           WINDOW          │                             │         DELIM_JOIN        │
│    ────────────────────   │                             │    ────────────────────   │
│        Expressions:       │                             │   Join Type: RIGHT_SEMI   │
│        delim_index        │                             │                           │
│                           │                             │        Conditions:        │
│                           │                             │   (union IS NOT DISTINCT  ├──────────────┐
│                           │                             │         FROM union)       │              │
│                           │                             │(fixed_nested_varchar_array│              │
│                           │                             │    IS NOT DISTINCT FROM   │              │
│                           │                             │ fixed_nested_varchar_array│              │
│                           │                             │             )             │              │
└─────────────┬─────────────┘                             └─────────────┬─────────────┘              │
┌─────────────┴─────────────┐                             ┌─────────────┴─────────────┐┌─────────────┴─────────────┐
│       CROSS_PRODUCT       │                             │         PROJECTION        ││       CROSS_PRODUCT       │
│    ────────────────────   │                             │    ────────────────────   ││    ────────────────────   │
│                           ├──────────────┐              │        Expressions:       ││                           ├──────────────┐
│                           │              │              │           union           ││                           │              │
│                           │              │              │ fixed_nested_varchar_array││                           │              │
└─────────────┬─────────────┘              │              └─────────────┬─────────────┘└─────────────┬─────────────┘              │
┌─────────────┴─────────────┐┌─────────────┴─────────────┐┌─────────────┴─────────────┐┌─────────────┴─────────────┐┌─────────────┴─────────────┐
│          SEQ_SCAN         ││          SEQ_SCAN         ││           FILTER          ││          SEQ_SCAN         ││         DELIM_GET         │
│    ────────────────────   ││    ────────────────────   ││    ────────────────────   ││    ────────────────────   ││    ────────────────────   │
│          my_table         ││          my_table         ││        Expressions:       ││          my_table         ││                           │
│                           ││                           ││   CAST(union AS BOOLEAN)  ││                           ││                           │
└───────────────────────────┘└───────────────────────────┘└─────────────┬─────────────┘└───────────────────────────┘└───────────────────────────┘
                                                          ┌─────────────┴─────────────┐
                                                          │         DELIM_GET         │
                                                          │    ────────────────────   │
                                                          └───────────────────────────┘

I've tried simplifying the query, but haven't managed to get it. I think the issue is lies here 

┌─────────────┴─────────────┐
│         DELIM_JOIN        │
│    ────────────────────   │
│   Join Type: RIGHT_SEMI   │
│                           │
│        Conditions:        │
│   (union IS NOT DISTINCT  ├──────────────┐
│         FROM union)       │              │
│(fixed_nested_varchar_array│              │
│    IS NOT DISTINCT FROM   │              │
│ fixed_nested_varchar_array│              │
│             )             │              │
└─────────────┬─────────────┘              │
┌─────────────┴─────────────┐┌─────────────┴─────────────┐
│         PROJECTION        ││       CROSS_PRODUCT       │
│    ────────────────────   ││    ────────────────────   │
│        Expressions:       ││                           ├──────────────┐
│           union           ││                           │              │
│ fixed_nested_varchar_array││                           │              │
└─────────────┬─────────────┘└─────────────┬─────────────┘              │
┌─────────────┴─────────────┐┌─────────────┴─────────────┐┌─────────────┴─────────────┐
│           FILTER          ││          SEQ_SCAN         ││         DELIM_GET         │
│    ────────────────────   ││    ────────────────────   ││    ────────────────────   │
│        Expressions:       ││          my_table         ││                           │
│   CAST(union AS BOOLEAN)  ││                           ││                           │
└─────────────┬─────────────┘└───────────────────────────┘└───────────────────────────┘
┌─────────────┴─────────────┐
│         DELIM_GET         │
│    ────────────────────   │
└───────────────────────────┘

Normally the CAST(union AS BOOLEAN) will throw a conversion error, but I think because it's from a DELIM GET the conversion doesn't happen properly (stepping through the code, I never get to the Filter during the execution phase).

The current error happens during the DELIM JOIN. If you put a breakpoint on vector_copy:229, the source_entry structure has impossible values for offset and length, leading me to believe the delim get on the right side and the left side are not properly propagating the types and the checks that normally would throw conversion errors are not throwing conversion errors now.

@lnkuiper Does this theory make sense to you? Maybe we can look into this issue together Wednesday? It's not critical as it only happens in debug mode.

lnkuiper commented 2 months ago

Took me a while, but the issue is that the cached cast vector for using ARRAYs in the TupleDataCollection was not reset properly when Gathering the vector in the JoinHashTable. I was able to fix this, and will send a PR

Tmonster commented 2 months ago

@lnkuiper I hope it wasn't too long. Thanks for finding the fix 🙏

lnkuiper commented 2 months ago

PR is up here https://github.com/duckdb/duckdb/pull/13708