No module named 'delta_kernel_rust_sharing_wrapper' on delta-sharing 1.1.0

[championj-db]

Issue After pip install delta-sharing I cannot run an import delta_sharing command as I receive this error: ModuleNotFoundError: No module named 'delta_kernel_rust_sharing_wrapper'

Reverting back to 1.0.5 with pip install delta-sharing==1.0.5 appears to resolve this issue, As does doing a pip install delta-kernel-rust-sharing-wrapper

Expectation Doing a pip install delta-sharing on the latest version should be able to pull in required dependencies

Context I am trying to run the delta-sharing-airlines demo https://www.databricks.com/resources/demos/tutorials/data-sharing/delta-sharing-airlines and encounter this in notebook 03-receiver-delta-sharing-demo using the generated cluster (DBR 12.2 LTS ML, Spark 3.3.2, Scala 2.12)

[forzagreen]

As explained in the release notes of v1.1.0, section Important Installation Notes, you must install delta-kernel-rust-sharing-wrapper

I agree that it would be better if delta-kernel-rust-sharing-wrapper were a dependancy of delta-sharing.

[GBBBAS]

As explained in the release notes of v1.1.0, section Important Installation Notes, you must install delta-kernel-rust-sharing-wrapper

I agree that it would be better if delta-kernel-rust-sharing-wrapper were a dependancy of delta-sharing.

Can we expect that it is it planned to be added as a dependency in a next release?

[GeroSalas]

still experiencing issue with this

here is my Dockerfile

FROM public.ecr.aws/lambda/python:3.9

# Verify that python3 is installed correctly
RUN python3 --version
RUN python3 -m pip install --no-cache-dir --upgrade pip
RUN python3 -m pip install pyodbc

# Update the package list and install glibc 2.31+ or higher
RUN yum -y update && yum -y install glibc
# Verify glibc version
RUN ldd --version

#WORKDIR ${LAMBDA_TASK_ROOT}

COPY index.py ${LAMBDA_TASK_ROOT}
COPY requirements.txt ${LAMBDA_TASK_ROOT}
COPY src ${LAMBDA_TASK_ROOT}/src

# Delta-Sharing Release Notes
# https://github.com/delta-io/delta-sharing?tab=readme-ov-file#installation
# https://github.com/delta-io/delta-sharing/releases/tag/py-v1.1.0
RUN python3 -m pip install delta-kernel-rust-sharing-wrapper

RUN python3 -m pip install --upgrade -r requirements.txt -t ${LAMBDA_TASK_ROOT}

CMD [ "index.handler" ]

[cvargas-xbrein]

same error from aws lambda python 3.10 when try to installing deps on efs storage

 ERROR: Could not find a version that satisfies the requirement delta-kernel-rust-sharing-wrapper (from delta-sharing) (from versions: none)
ERROR: No matching distribution found for delta-kernel-rust-sharing-wrapper

[xy1024xiangyu]

same error here. Also tried to install from the whl file manually, but seems currently not supporting AWS EC2 x86_64 instance?

also tried to follow this https://github.com/delta-io/delta-sharing/blob/main/python/delta-kernel-rust-sharing-wrapper/README.md cannot figure out a solution...

[mattaop]

I had the same issue when running spark 3.3, but when running spark 3.4 the issue disappeared. Then delta-kernel-rust-sharing-wrapper was installed as part of delta-sharing