search

Shopify / pyoozie

Library for querying and scheduling with Apache Oozie
https://py-oozie.readthedocs.io
MIT License
11 stars 12 forks source link

Split string validation into names or identifiers and enforce #15

Closed cfournie closed 7 years ago

cfournie commented 7 years ago

Testing with Oozie 4.1.0 shows that identifiers (e.g. workflow action names) need to follow the regex ^[a-zA-Z_][\\-_a-zA-Z0-9]{0,38}$, however names (e.g. workflow and coordinator names) do not need to adhere to that regex (e.g. can contain spaces/punctuation) but are limited to 255 chars by Oozie's backing database and are limited to printable ASCII characters.

Creating a workflow with a really long action name results in:

org.apache.oozie.command.CommandException: E0724: Invalid node name, name [action-ppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp] must be 50 chars or less
        at org.apache.oozie.command.wf.SubmitXCommand.execute(SubmitXCommand.java:272)

Creating a workflow with a very long name results in:


2017-02-27 17:35:36,418 ERROR SubmitXCommand:538 - SERVER[oozie] USER[root] GROUP[-] TOKEN[] APP[aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa] JOB[0000019-170227162353067-oozie-root-W] ACTION[-] XException,
org.apache.oozie.command.CommandException: E0803: IO error, E0603: SQL error in operation, <openjpa-2.2.2-r422266:1468616 fatal store error> org.apache.openjpa.persistence.RollbackException: The transaction has been rolled back.  See the nested exceptions for details on the errors that occurred.
FailedObject: org.apache.oozie.WorkflowJobBean@27d6d59
...
Caused by: <openjpa-2.2.2-r422266:1468616 fatal general error> org.apache.openjpa.persistence.PersistenceException: A truncation error was encountered trying to shrink VARCHAR 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa&' to length 255.
cfournie commented 7 years ago

@honkfestival @kmtaylor-github this should add the necessary safety for names and IDs while being more permissive than before.

cfournie commented 7 years ago

@honkfestival @kmtaylor-github now this even passes CI. 👍 /👎 ?