search

jupyter-server / enterprise_gateway

A lightweight, multi-tenant, scalable and secure gateway that enables Jupyter Notebooks to share resources across distributed clusters such as Apache Spark, Kubernetes and others.
https://jupyter-enterprise-gateway.readthedocs.io/en/latest/
Other
615 stars 221 forks source link

Start EG in demo-image to inherit PYSPARK_PYTHON env #1230

Closed kevin-bates closed 1 year ago

kevin-bates commented 1 year ago

I suspect the change to not unconditionally inherit the value of env PATH has side-affected the CI test test_r_kernel.py::TestRKernelClient because it can no longer find the python command (based on troubleshooting the issue).

Rather than flow PATH via the inherited_envs configurable (as suggested here), this change updates the start script of the enterprise-gateway-demo image to add the value of env PYSPARK_PYTHON via inherited_envs since that's the minimal value necessary to get the TestRKernelClient test passing. It also serves as an example of how to propagate a value from the EG process to kernels.

kevin-bates commented 1 year ago

Thank you for your review @luong-komorebi - much appreciated!