Delta Lake: Up and Running, missing instructions in repo readme

[alexmahrou]

Hi @benniehaelen ,

I'm following along in the Delta Lake: Up and Running book, and after pulling down the image, and running the image, your instructions say to perform a pip-install delta-spark. However this yields pip: command not found. I believe there are some steps in-between on the environment setup that would be helpful to provide. I think when you mention "I have included several instructions on how to work with the container in the readme of the book's GitHub repository", then link to this repo, the intention was to have those steps there. But I don't see anything in the readme at this time. Am I missing something?

Thanks!

Alex

[mattdajacob]

I'm having the same issue. No instructions in the README. The pip install delta-spark initiates for me, but errors out: ``` "ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/home/spark' Check the permissions."

[AmerM137]

same error as @mattdajacob, any way we can get some help on this please?

[benniehaelen]

@mattdajacob and @AmerM137. Have you tried adding --user to the pip command? The error is likely because you don't have admin rights on the machine. The "--user" flag installs just for the current user. Thanks!

[benniehaelen]

See my reply, let me know if you still have issues. Every local system is always a bit different regarding permissions etc... so it's hard to address every permutation.

Thanks for buying the book!

Bennie

On Wed, Nov 8, 2023 at 9:20 PM AmerM137 @.***> wrote:

same error as @mattdajacob https://github.com/mattdajacob, any way we can get some help on this please?

— Reply to this email directly, view it on GitHub https://github.com/benniehaelen/delta-lake-up-and-running/issues/5#issuecomment-1803139469, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJHEYSW5VEPLSDBYZIHCKLYDRKY7AVCNFSM6AAAAAATLKKPTCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBTGEZTSNBWHE . You are receiving this because you were mentioned.Message ID: @.***>

[AmerM137]

Thanks @benniehaelen but that did not work unfortunately. I've had much better luck with the official delta-docker image.

https://hub.docker.com/r/deltaio/delta-docker https://github.com/delta-io/delta-docs/tree/main/static/quickstart_docker

Pinging @mattdajacob and @alexmahrou so they can see this too. Hope this helps! I am up and running smoothly now.

[coldavidjbrady]

I had the same issue reported by the folks above. My thanks to @AmerM137 for the links to delta-docker. I am also up and running now, but it would be good to get the issues resolved with installing delta-spark following the instructions in the book.

[FdR-Incubator]

I also need to offer thanks to @AmerM137 for the hint. I also found following link useful in terms of starting the interactive pyspark shell session (as well as scala and some others) https://github.com/delta-io/delta-docker#pyspark-shell