search

trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
10.15k stars 2.93k forks source link

Trino router not compatible with delta table using rust engine #22842

Open gprashmi opened 1 month ago

gprashmi commented 1 month ago

We have a delta table created using delta-rs with Rust engine and we want to map this table to TrinoRouter for Grafana integration. However when we try to map this to TrinoRouter, it shows compatibility issues with the error.

Caused by: com.fasterxml.jackson.databind.exc.ValueInstantiationException: Cannot construct instance of io.trino.plugin.deltalake.transactionlog.AddFileEntry, problem: Could not parse JSON at [Source: (String)"

Trino also not able to read any new data written into the delta-table using Rust.

However, when we try the same with Pyarrow engine, we are able to map the table to Trino successfully. But eventually we want to have the table written using Rust engine. Can you please let me know if there is a new version of Trino that is compatible with rust delta tables or is there any other way to resolve this issue?

gprashmi commented 1 month ago

@g12-al

ebyhr commented 1 month ago

@gprashmi Please share the problematic transaction log after masking confidential info.