search

duckdb / duckdb_delta

DuckDB extension for Delta Lake
MIT License
107 stars 13 forks source link

failed to read delta table which stored by OSS ( Aliyun's Object Storage Service) through S3 SDK #54

Open Mr-j-yangyu opened 1 month ago

Mr-j-yangyu commented 1 month ago

Client Info

version: v1.0.0 paltform: macOS

Query:

CREATE PERSISTENT SECRET st ( TYPE S3, KEY_ID '', SECRET '', REGION 'oss-cn-hangzhou', ENDPOINT 's3.oss-cn-hangzhou.aliyuncs.com' );

SELECT * FROM delta_scan('s3://91jkys-dataworks-qa/delta-table/origin.db/ods_user_2');

Result

IO Error: Hit DeltaKernel FFI error (from: While trying to read from delta table: 's3://91jkys-dataworks-qa/delta-table/origin.db/ods_user_2/'): Hit error: 8 (ObjectStoreError) with message (Error interacting with object store: Generic S3 error: Error after 10 retries in 4.576841894s, max_retries:10, retry_timeout:180s, source:error sending request for url (https://s3.oss-cn-hangzhou.amazonaws.com/91jkys-dataworks-qa/delta-table/origin.db/ods_user_2/_delta_log/_last_checkpoint))

Mr-j-yangyu commented 1 month ago

It seems that the error was caused by ENDPOINT is changed to 's3.oss-cn-hangzhou.amazonaws.com'( not configurated by myself )

samansmink commented 3 weeks ago

@Mr-j-yangyu Could you retry with the latest nightly build of delta using force install delta from core_nightly? I think this should be fixed now