Не работают запросы 51 и 97 с MapJoin (или с CBO)

ulya-sidorina commented 4 months ago

2024-06-25 08:31:07.792 INFO  dqrun(pid=3916800, tid=0x00007FBD9FB20C00) [default] yql_facade.cpp:1391: {dummy_op} Fallback skipped due to per query policy
./yql/ds/q51.sql:<main>: Error: Type annotation

    ./yql/ds/q51.sql:<main>:67:1: Error: At function: TopSort
        limit 100;
        ^
        ./yql/ds/q51.sql:<main>:49:7: Error: At function: FlatMap
                from (select item_sk
              ^
            ./yql/ds/q51.sql:<main>:57:12: Error: At function: CalcOverWindow, At function: FlatMap
                     from (select case when web.item_sk is not null then web.item_sk else store.item_sk end item_sk
                   ^
                ./yql/ds/q51.sql:<main>:61:28: Error: At function: DqCnUnionAll, At function: TDqOutput, At function: DqStage, At function: NarrowMap, At function: ExpandMap, At function: FlatMap, At function: FlatMap, At function: JoinDict
                                   from $web_v1 as web full outer join $store_v1 as store on (web.item_sk = store.item_sk
                                   ^
                    ./yql/ds/q51.sql:<main>:61:28: Error: Expected dry type, but got: Tuple<Int32?,Date?>
                                   from $web_v1 as web full outer join $store_v1 as store on (web.item_sk = store.item_sk

ulya-sidorina commented 4 months ago

Simplified repro:

$web_v1 = (
  select web_sales.ws_item_sk item_sk, date_dim.d_date as d_date from bindings.web_sales as web_sales
      cross join bindings.date_dim as date_dim
          where ws_sold_date_sk=d_date_sk
);

$store_v1 = (
    select store_sales.ss_item_sk item_sk, date_dim.d_date as d_date from bindings.store_sales as store_sales
        cross join bindings.date_dim as date_dim
             where ss_sold_date_sk=d_date_sk
);

select * from $web_v1 as web full outer join $store_v1 as store on (web.item_sk = store.item_sk and web.d_date = store.d_date);

ulya-sidorina commented 4 months ago

Query fails here: https://github.com/ydb-platform/ydb/blob/16981dcdd4d003bb698482b0544034bc0f55e2dd/ydb/library/yql/core/type_ann/type_ann_join.cpp#L177

CBO chose EJoinAlgoType::MapJoin , but JoinDict was built here: https://github.com/ydb-platform/ydb/blob/16981dcdd4d003bb698482b0544034bc0f55e2dd/ydb/library/yql/dq/opt/dq_opt_join.cpp#L1479

ulya-sidorina commented 4 months ago

We should deprecate using EHashJoinMode::Map for full and exclusion joins

ydb-platform / ydb

Не работают запросы 51 и 97 с MapJoin (или с CBO) #6019