astronomer / airflow-provider-duckdb

A provider package for DuckDB
Apache License 2.0
14 stars 3 forks source link

Unable to use HTTPFS extension in task #11

Open hrm13 opened 1 year ago

hrm13 commented 1 year ago

When attempting to use DuckDB v0.9.0 with HTTPS to retrieve a parquet file in S3 I receive the following error:

 File "/usr/local/airflow/dags/xxx/xxx.py", line 58, in xxx
    conn.sql("INSTALL httpfs")
duckdb.duckdb.HTTPException: HTTP Error: Failed to download extension "httpfs" at URL "http://extensions.duckdb.org/v0.9.0/linux_arm64_gcc4/httpfs.duckdb_extension.gz"
Extension "httpfs" is an existing extension.
Are you using a development build? In this case, extensions might not (yet) be uploaded.

I've tried using the DuckDB library directly (instead of having it installed through the provider, though I assume this doesn't make a difference anyway).

It works just fine on my local machine (macOS), which makes me suspect the issue is with the download link mentioned above. When I try to go to this myself, I get an access denied error.

Other relevant information: