Rather than have the pod creator set the SPARK_CONF_DIR environment
variable to point to a directory with a modified spark configuration, this change
simply copies the contents of $UPDATE_SPARK_CONF_DIR to $SPARK_HOME/conf
during startup. This gives us a simple overwrite semantic so that only files being
changed must be specified.
Rather than have the pod creator set the SPARK_CONF_DIR environment variable to point to a directory with a modified spark configuration, this change simply copies the contents of $UPDATE_SPARK_CONF_DIR to $SPARK_HOME/conf during startup. This gives us a simple overwrite semantic so that only files being changed must be specified.