search

exacaster / lighter

REST API for Apache Spark on K8S or YARN
MIT License
91 stars 21 forks source link

use default value for LIGHTER_BATCH_DEFAULT_CONF #962

Closed cometta closed 6 months ago

cometta commented 6 months ago

can LIGHTER_BATCH_DEFAULT_CONF use same value of LIGHTER_SESSION_DEFAULT_CONF so that no need to duplicate same value for both ? Otherwise, i would need duplicate same values for both key

pdambrauskas commented 6 months ago

In most cases these values are different. Things like default resources & default docker image usually differ between sessions and batch applications.

If you are using Helm you can parametrise ENV variables through values file and use the same value for both environment variables.