search

delta-io / delta-sharing

An open protocol for secure data sharing
https://delta.io/sharing
Apache License 2.0
770 stars 172 forks source link

delta-sharing package issue in pyspark #325

Open param422 opened 1 year ago

param422 commented 1 year ago

Hi Team,

Requirement: Install delta_sharing package in one of our EMR server.

We have installed the package in python using the below command and we are able to se and access the package.

pip install delta-sharing

Now, the issue is when we try to import this package from pyspark (jupyter notebook) it is giiving No Module error.

Below is the screenshot of the error.

image

Can you please assit on this as this is critical to our project.

param422 commented 1 year ago

Could you please help us on this issue.

param422 commented 1 year ago

hi team, could you please help me on this issue.

linzhou-db commented 1 year ago

@param422 Sorry for the delay, taking a look.

zsxwing commented 1 year ago

@param422 Could you try some other python library? I feel it's likely you are using the right way to install python libraries for jupyter notebook.

I found this post from EMR: https://aws.amazon.com/cn/blogs/big-data/install-python-libraries-on-a-running-cluster-with-emr-notebooks/

Could you try above and see if it works?