search

metatron-app / metatron-discovery

Powerful & Easy way for big data discovery
https://metatron.app
Apache License 2.0
440 stars 110 forks source link

Allow building bloom filter on broadcasted alias #4302

Open navis opened 11 months ago

navis commented 11 months ago

TPCH-9

04:10:52.446 - -> INNER (supplier:20000:1.000 + nation:25:1.000)
04:10:52.446 - -- nation:25:1.000 (R) will be broadcasted to supplier (L)
04:10:52.535 - -- nation (R) is materialized (25 rows)
04:10:52.535 - -> INNER (lineitem:7823529:1.000 + part:10818:0.054)
04:10:52.573 - -- part (R) is materialized (10818 rows)
04:10:52.582 - --- selectivity 0.054 merged into lineitem(7823529:1.000 to 423174:0.054)
04:10:52.582 - -- part:10818:0.054 (R) is merged into lineitem (L) as filter on [L_PARTKEY]
04:10:52.582 - -> INNER (lineitem+part:423174:0.065 + supplier+nation:20000:1.000)
04:10:52.582 - -- supplier+nation:20000:1.000 (R) will be broadcasted to lineitem+part (L)
04:10:52.638 - -- supplier+nation (R) is materialized (10000 rows)
04:10:52.642 - -> INNER (partsupp:800000:1.000 + lineitem+part+supplier+nation:423174:0.078)
04:10:52.642 - -- partsupp:800000:1.000 (L) (sort)
04:10:52.642 - -- lineitem+part+supplier+nation:423174 (R) (sort)
04:10:52.642 - -- .. with bloom filter $view:lineitem[L_SUPPKEY, L_PARTKEY](InDimFilter{dimension='L_PARTKEY', values=[100001, 10001, 100043, 100059, 100072, 100075, 100076, 100080, 1001, 100115, ..10808 more]}) (R) to partsupp:800000:1.000 (L)
04:10:52.642 - --- selectivity 0.078 merged into partsupp(800000:1.000 to 62311:0.078)
04:10:52.646 - -> INNER (orders:2086113:1.000 + partsupp+lineitem+part+supplier+nation:423174:0.093)
04:10:52.646 - -- orders:2086113:1.000 (L) (sort)
04:10:52.646 - -- partsupp+lineitem+part+supplier+nation:423174 (R) (sort)
04:10:52.651 - --- compressed in filter [L_PARTKEY:10818], 41,599 bytes into 28,093 bytes (32% reduction, 4 msec)
04:10:52.815 - Running 2-way join processing [partsupp, lineitem+part+supplier+nation]
04:10:53.427 - >> INNER ([partsupp].[PS_SUPPKEY, PS_PARTKEY](sorted-stream:62311?) --> [lineitem+part+supplier+nation].[L_SUPPKEY, L_PARTKEY](sorted-stream:423174?)) (SortedMerge)
04:10:53.432 - Running 2-way join processing [orders, partsupp+lineitem+part+supplier+nation]
04:10:54.069 - << INNER ([partsupp].[PS_SUPPKEY, PS_PARTKEY](sorted-stream:84) + [lineitem+part+supplier+nation].[L_SUPPKEY, L_PARTKEY](sorted-stream:323447)), resulting 68 rows ([PS_PARTKEY, PS_SUPPKEY, PS_SUPPLYCOST]+[L_DISCOUNT, L_EXTENDEDPRICE, L_ORDERKEY, L_PARTKEY, L_QUANTITY, L_SUPPKEY, N_NAME]) : (641 msec)
04:10:54.069 - >> INNER ([orders].[O_ORDERKEY](sorted-stream:2086113?) --> [partsupp+lineitem+part+supplier+nation].[L_ORDERKEY](sorted-stream:68?)) (SortedMerge)
04:10:54.645 - << INNER ([orders].[O_ORDERKEY](sorted-stream:1495711) + [partsupp+lineitem+part+supplier+nation].[L_ORDERKEY](sorted-stream:68)), resulting 67 rows ([O_ORDERKEY, v0]+[PS_SUPPLYCOST, L_DISCOUNT, L_EXTENDEDPRICE, L_ORDERKEY, L_QUANTITY, N_NAME]) : (575 msec)
04:10:54.646 - {"success":true,"query/time":2390,"query/rows":36,"query/bytes":2401}    "SELECT /* TPCH9 */    NATION,    O_YEAR,    SUM(AMOUNT) AS SUM_PROFIT FROM    (        SELECT            N_NAME AS NATION,            YEAR(O_ORDERDATE) AS O_YEAR,            L_EXTENDEDPRICE * (1 - L_DISCOUNT) - PS_SUPPLYCOST * L_QUANTITY AS AMOUNT        FROM            part,            supplier, (..347 more..) AND PS_SUPPKEY = L_SUPPKEY            AND PS_PARTKEY = L_PARTKEY            AND P_PARTKEY = L_PARTKEY            AND O_ORDERKEY = L_ORDERKEY            AND S_NATIONKEY = N_NATIONKEY            AND P_NAME LIKE '%plum%'    ) AS PROFIT GROUP BY    NATION,    O_YEAR ORDER BY    NATION,    O_YEAR DESC"