A lightweight, multi-tenant, scalable and secure gateway that enables Jupyter Notebooks to share resources across distributed clusters such as Apache Spark, Kubernetes and others.
This pull request applies similar logic for flowing envs to spark-operator pods to that used by regular pods. The envs are added into the container's specification if the CRD's resource group is sparkoperator.k8s.io. Both the driver and executor envs are updated. If the group is not sparkoperator.k8s.io, nothing occurs since, presumably, the location of the env stanza will be different.
While troubleshooting this change, it was discovered that the spark operator's webhook server must be enabled for envs to be transferred. As a result, both the deployment document and the spark application jinja template have been updated with the appropriate helm command.
Since the doc update uses a nested Note/Tip box that looks a bit strange in the raw text, I'm including a screenshot here:
I have confirmed that envs listed in the kernel.json's env stanza are present within the notebook, as are allKERNEL_-prefixed envs.
This pull request applies similar logic for flowing envs to spark-operator pods to that used by regular pods. The envs are added into the container's specification if the CRD's resource group is
sparkoperator.k8s.io. Both the driver and executor envs are updated. If the group is notsparkoperator.k8s.io, nothing occurs since, presumably, the location of the env stanza will be different.While troubleshooting this change, it was discovered that the spark operator's
webhook servermust be enabled for envs to be transferred. As a result, both the deployment document and the spark application jinja template have been updated with the appropriatehelmcommand.Since the doc update uses a nested Note/Tip box that looks a bit strange in the raw text, I'm including a screenshot here:
I have confirmed that envs listed in the
kernel.json'senvstanza are present within the notebook, as are allKERNEL_-prefixed envs.Resolves #1224