search

delta-io / delta-docker

Official Dockerfile for Delta Lake
36 stars 16 forks source link

updated to delta-spark 3.5.1, added tree #8

Closed tristenwentling closed 6 months ago

tristenwentling commented 7 months ago

Resolves issue #7

BASE_CONTAINER updated to spark:3.5.1-scala2.12-java17-python3-ubuntu

Docker image was created and tested with the versions of following packages.

DELTA_SPARK_VERSION="3.1.0" DELTALAKE_VERSION="0.16.4" JUPYTERLAB_VERSION="4.0.7" PANDAS_VERSION="2.2.2" ROAPI_VERSION="0.11.1"

Built the image and tested the quickstart notebook and everything runs.

Tested my DV code notebook on the same image and everything runs now with the updated Delta Lake version.