Load configuration files eagerly during run-transient

[jbeemster]

If the emr_cluster config path is valid and the cluster launches but the playbook path is not available then the command exits - leaving the cluster running for no reason.

Either we need to eagerly evaluate all assets (emr config and playbook config) or we need to kill the EMR cluster on playbook resolution failure.

https://github.com/snowplow/dataflow-runner/blob/master/src/main.go#L236

cc/ @alexanderdean

[jbeemster]

Actually sorry the above is not quite right - the issue is that the config is never loaded into memory for either. So if the config is removed mid run then dataflow-runner cannot perform playbook or "down" function.

It would be good to eagerly evaluate and load the assets into memory at start to avoid this potential issue.

[alexanderdean]

Ouch, good catch!

[jbeemster]

@BenFradet do you have any idea on when this will be mile-stoned to be fixed?

[BenFradet]

This should be in the sprint starting nov 27th :+1: