search

trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
9.85k stars 2.85k forks source link

Queries on hudi tables are hanging for indefinite time after certain load #20291

Open ryadav-uptycs opened 5 months ago

ryadav-uptycs commented 5 months ago

Observation:

if we fire 10-12 hudi table queries in parallel on trino cluster (430) then queries are never finishing , if running simple limit 1 query is not finishing .

We are upgrading from trino 418 to 430 . in 418 version everything works fine , with same amount of load, queries are finishing in trino 418 where with trino 430 they are hanging .

During low load it is working fine .

Attaching screenshot of the issue

query running from last 11 mins and still not completing query

trino UI

Screenshot from 2024-01-08 10-46-56

Each worker node size trino

alberttwong commented 6 days ago

It's probably because they changed the Hudi connector in Trino 419. Trino 418 is the last working version for most Hudi versions. https://github.com/trinodb/trino/pull/18840#issuecomment-1772069499