search

snowplow / dataflow-runner

Run templatable playbooks of Hadoop/Spark/et al jobs on Amazon EMR
http://snowplowanalytics.com
19 stars 8 forks source link

Short-circuit execution on unset template variable #16

Closed chuwy closed 7 years ago

chuwy commented 7 years ago

If user didn't set some template variable via --vars, step is still running with "<no value>" as a value, which in most cases not what user wants, but default text/template behavior.

I think we should instead short-circuit execution of dataflow-runner. If user wants to leave it blank he may use whole text/template template language.

alexanderdean commented 7 years ago

Agree!

chuwy commented 7 years ago

Problem with template language though is that it adds double quotes.

BenFradet commented 7 years ago

Does it make sense to short-circuit for unset env variables with the systemEnv template function?

BenFradet commented 7 years ago

^ what do you think @chuwy ?

chuwy commented 7 years ago

Yep. That's how I'd expect it to work.