Now, instead of creating new intermediate objects (YamlWorkflow, YamlJob, and YamlProject), the YamlCompiler is used to parse + output yaml directly from Workflow and Job objects.
This fits in much more closely with how the AzkabanDslCompiler works and allows for less hacky-ness when trying to make the YamlCompiler work with the LiBangBangJob which is required to roll-out at LinkedIn.
The changes themselves are basically just aggregating the logic from the Yaml* objects and their tests into the YamlCompiler itself.
Also includes one small change involving the cleanBuildDirectory step in the AzkabanDslCompiler and YamlCompiler objects where they now delete both .job/.properties AND .flow/.project files. This is required to painless switch between compilers as we believe our users might try to do when testing Flow 2.0.
In terms of testing, no adjustments were required in hadoop-plugin-test in order for this to work and only minor tweaks were needed for the tests that were originally in YamlWorkflowTest and YamlJobTest.
Significant refactor of Flow 2.0 in DSL.
Now, instead of creating new intermediate objects (YamlWorkflow, YamlJob, and YamlProject), the YamlCompiler is used to parse + output yaml directly from Workflow and Job objects.
This fits in much more closely with how the AzkabanDslCompiler works and allows for less hacky-ness when trying to make the YamlCompiler work with the LiBangBangJob which is required to roll-out at LinkedIn.
The changes themselves are basically just aggregating the logic from the Yaml* objects and their tests into the YamlCompiler itself.
Also includes one small change involving the cleanBuildDirectory step in the AzkabanDslCompiler and YamlCompiler objects where they now delete both .job/.properties AND .flow/.project files. This is required to painless switch between compilers as we believe our users might try to do when testing Flow 2.0.
In terms of testing, no adjustments were required in hadoop-plugin-test in order for this to work and only minor tweaks were needed for the tests that were originally in YamlWorkflowTest and YamlJobTest.