duckdb / duckdb_delta

DuckDB extension for Delta Lake
MIT License
88 stars 8 forks source link

delta_scan cannot read from delta tables starting with uppercase letter. #7

Open mrjsj opened 1 month ago

mrjsj commented 1 month ago

Platform: MacOS ARM64

DuckDB version: 0.10.3

Trying to read a delta tabel with capital letter results in error:

D select * from delta_scan('file:///Users/jsj/Downloads/Pipelines');
IO Error: Hit DeltaKernel FFI error (from: snapshot in DeltaScanScanBind): Hit error: 15 (MissingVersionError) with message (No table version found.)

It works fine with the following delta table names:

mrjsj commented 1 month ago

Update:

It works if I put a slash after the table name, e.g.

select * from delta_scan('file:///Users/jsj/Downloads/Pipelines/');

I have tried making the unit tests fail in delta-kernel-rs by making the the delta tables upper-case, but no issue there (or I'm not proficient enough in Rust).

For the record - the name of the delta table itself (on disk) doesn't matter, but if the delta table specified in the query is capitalized it fails.

samansmink commented 1 month ago

Im suspecting this to be a duplicate of https://github.com/duckdb/duckdb_delta/issues/17 meaning that this is also fixed by https://github.com/duckdb/duckdb_delta/pull/20.

The problem is that delta-kernel-rs needs paths to end in a / otherwise weird stuff happens. Now DuckDB will internally make sure that the paths passed to kernel are always ending in a /

mrjsj commented 1 month ago

Great, thank you!