As I understand it, there is a config_overrides = {} piece of code that will always override all existing configs at the app level. Consequently, I assume that from the UI, for example, while cloning or starting a new job, only extra "Spark properties" might be provided for the job run.
From your point of view: is this currently supported? And if not, what's the easiest way to implement it? Perhaps an additional call to the GetApplication API to fetch that information. Ideally, I want to have some common-ground dependency and not have to worry about remembering to pass it every time
Hi there,
As I understand it, there is a config_overrides = {} piece of code that will always override all existing configs at the app level. Consequently, I assume that from the UI, for example, while cloning or starting a new job, only extra "Spark properties" might be provided for the job run.
From your point of view: is this currently supported? And if not, what's the easiest way to implement it? Perhaps an additional call to the GetApplication API to fetch that information. Ideally, I want to have some common-ground dependency and not have to worry about remembering to pass it every time