Unable to Read Delta Table due to extension download problems

ZadeFrontier commented 4 months ago

What happens?

I tried reading delta table using delta_scan but i got the below error that the delta extension is unable to download for installation into duckdb. When i tried to access the link directly, the access is denied. Please help. I am currently unable to read delta tables on my delta lake.

Traceback (most recent call last): File "c:\Users\RnD\test_delta_scan.py", line 16, in con.execute("Install delta") duckdb.duckdb.HTTPException: HTTP Error: Failed to download extension "delta" at URL "http://extensions.duckdb.org/v0.10.3/windows_amd64/delta.duckdb_extension.gz"

Candidate extensions: "md", "inet"

To Reproduce

[hobbies.zip](https://github.com/duckdb/duckdb/files/15444009/hobbies.zip)

import duckdb

# Connect to DuckDB
con = duckdb.connect()

# Define the path to your local Delta table
delta_table_path = 'file:///C:/home/data/deltalake/delta_tables/hobbies'

# Execute the delta_scan function to read the Delta table
query = f"""
SELECT * FROM delta_scan('{delta_table_path}')
"""

# Run the query and fetch the results
result = con.execute("SELECT extension_name, installed, description FROM duckdb_extensions();").fetchdf()
con.execute("Install delta")

# Print the results
print(result)

OS:

Windows 11 x64

DuckDB Version:

0.10.3

DuckDB Client:

Python 3.11.7

Full Name:

Krishna Cheedella

Affiliation:

Keysight Technologies

What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.

I have tested with a stable release

Did you include all relevant data sets for reproducing the issue?

Yes

Did you include all code required to reproduce the issue?

[X] Yes, I have

Did you include all relevant configuration (e.g., CPU architecture, Python version, Linux distribution) to reproduce the issue?

[X] Yes, I have

carlopi commented 4 months ago

The REAMDE lists the currently supported platforms, and windows_amd64 is not there.

Consider subscribe/commenting on #10 that should cover this problem.

djouallah commented 4 months ago

using duckdb-0.10.4.dev55

I still get the same problem

<timed exec> in <module>

[/usr/local/lib/python3.10/dist-packages/duckdb/__init__.py](https://localhost:8080/#) in sql(query, **kwargs)
    455     else:
    456         conn = duckdb.connect(":default:")
--> 457     return conn.sql(query, **kwargs)
    458 _exported_symbols.append('sql')
    459 

Error: An error occurred while trying to automatically install the required extension 'delta':
Failed to download extension "delta" at URL "http://extensions.duckdb.org/73d615ea83/linux_amd64_gcc4/delta.duckdb_extension.gz"

Candidate extensions: "md", "inet"

carlopi commented 4 months ago

I think delta is, at the moment, only available for v0.10.3 plus selected nightly version, but it's not there in general.

Release process for delta is currently triggered manually, and has not been (yet) moved into main CI workflows.

carlopi commented 4 months ago

This is also sort of explicit with this rewording of the README: https://github.com/duckdb/duckdb_delta/pull/15/files

samansmink commented 4 months ago

delta will (for now) not be available in DuckDB's nightly builds because of the extra toolchain setup is needs to compile the rust dependency.

However nightly binaries of the delta extension itself (targeting the latest stable duckdb) will be pushed automatically to the core_nightly repository.

I will close this issue in favor of the other one

duckdb / duckdb_delta