search

Giorgi / DuckDB.NET

Bindings and ADO.NET Provider for DuckDB
https://duckdb.net
MIT License
411 stars 69 forks source link

Aws S3 support #21

Closed ravi27may closed 1 year ago

ravi27may commented 2 years ago

Hi,

Am trying to read the parquet file data from aws s3 (similar to specified in the link provided https://github.com/duckdb/duckdb/blob/1fbc35f37e2b5b5e641ae0e0cb145078621c0c7b/test/sql/copy/parquet/test_parquet_remote.test) but it says the file didn't found.

Could someone please help how to read the data from aws s3 file

Giorgi commented 2 years ago

Does the same work when you run it from duckdb shell?

ravi27may commented 2 years ago

Yes Giorgi, it worked using the shell but in linux

Giorgi commented 2 years ago

@ravi27may Which version of duckdb.dll did you try it with?

Giorgi commented 2 years ago

@ravi27may To load files from HTTPS or S3 DuckDb needs to be compiled with HTTPFS extension: https://github.com/duckdb/duckdb/issues/1457#issuecomment-983606089

ravi27may commented 2 years ago

Hi Giorgi,

Am using the 0.3.1 version. so is this version already have httfs extension included any idea

Giorgi commented 2 years ago

I don't think 0.3.1 has BUILD_HTTPFS_EXTENSION=1

ravi27may commented 2 years ago

@Giorgi - how can we build the with httpfs extension to duckdb.dll. Could you please help me

Giorgi commented 2 years ago

That question belongs to the DuckDB repo

karthikmallireddy commented 1 year ago

@Giorgi Since duckdb 0.5.1 is released, if possible can you provide us an example with s3 file reading?

mikeTWC1984 commented 1 year ago

I think DuckDB never includes httpfs in their precompiled binaries. It can be installed by executing "INSTAL httpfs" sql command (it will download extension ) and then "LOAD httpfs".

Giorgi commented 1 year ago

Closing as this issue isn't related to .Net bindings.