Add workflow XML validation to test

cfournie commented 7 years ago

This PR adds the ability to validate that the workflow XML that's generated against the workflow XML schema using oozie validate (for which the minimum set of binaries required are provided via make install).

Violations fail the test and if verbosity is turned on will print the XML that was tested and the validation error from the script, e.g.:

        except subprocess.CalledProcessError as e:
            raise AssertionError('An XML validation error\n\n{error}\n\noccurred while parsing:\n\n{xml}'.format(
                error=e.output.decode('utf8').strip(),
>               xml=actual_xml,
            ))
E           AssertionError: An XML validation error
E
E           Error: Invalid app definition, org.xml.sax.SAXParseException; lineNumber: 5; columnNumber: 29; cvc-complex-type.2.4.a: Invalid content was found starting with element 'bad'. One of '{"uri:oozie:workflow:0.5":decision, "uri:oozie:workflow:0.5":fork, "uri:oozie:workflow:0.5":join, "uri:oozie:workflow:0.5":kill, "uri:oozie:workflow:0.5":action, "uri:oozie:workflow:0.5":end}' is expected.
E
E           occurred while parsing:
E
E           <?xml version="1.0" encoding="UTF-8"?>
E           <workflow-app xmlns="uri:oozie:workflow:0.5"
E                         name="descriptive-name">
E               <start to="action-payload" />
E               <action name="action-payload">
E                   <shell xmlns="uri:oozie:shell-action:0.3"><exec>echo "test"</exec></shell>
E                   <ok to="end" />
E                   <error to="action-error" />
E               </action>
E               <action name="action-error">
E                   <email xmlns="uri:oozie:email-action:0.2"><to>person@example.com</to><subject>Error</subject><body>A bad thing happened</body></email>
E                   <ok to="kill" />
E                   <error to="kill" />
E               </action>
E               <kill name="kill">
E                   <message>Failure message</message>
E               </kill>
E               <end name="end" />
E           </workflow-app>

tests/pyoozie/test_builder.py:153: AssertionError

To make this work, we need to download some JARs on make install. I've placed them in lib/* to match what the Oozie distribution looks like (those JARs are usually located in <oozie location>/lib).

review/ @kmtaylor-github @JasonMWhite

kmtaylor-github commented 7 years ago

I'm not sure what the protocol is, but is there a standard way to indicate this is a 3rd party package? Just dropping it in a top-level oozie folder makes it seem like code.

cfournie commented 7 years ago

is there a standard way to indicate this is a 3rd party package?

A lib folder might be a better way to go.

JasonMWhite commented 7 years ago

One comment, otherwise LGTM

JasonMWhite commented 7 years ago

👍

Shopify / pyoozie

Add workflow XML validation to test #12