WfSpec Custom Validations

[diablouma]

Oftentimes, we have variables in a WfSpec as follows:

WfRunVariable emailVar = wf.addVariable("email", VariableType.STR).required();
WfRunVariable customerId = wf.addVariable("customer-id", VariableType.STR).required();

We might want to enforce two validations:

1. For the email variable, we might want to require that it follows the "email" regex.
2. For the customer-id variable, we might want to require that the customer-id references a valid record in our customer database (external to LittleHorse).

There are additional types of validations that we might want to do, such as ensuring that a certain combination of variables is unique (i.e. a userId needs to be unique within an organization but not globally unique, so we can't use it as a WfRunId to enforce uniqueness).

Currently, the only way to enforce those validations is to do it before making the call to RunWf. I believe that we can improve this. Some ideas are as follows:

For the first item, which is a simple email validation:

• Introduce the concept of a schema that sits alongside the VariableType enum inside the VariableDef structure.
• Enforce schema compatibility in the RunWfRequestModel#process() method.

The second type, where we need to check with external systems, is slightly more complex. However I think we could introduce a new workflow concept like a ValidationTaskRun which is part of the ENTRYPOINT Node. It would work as follows:

• User specifies a ValidationTaskDefName which is part of the ENTRYPOINT Node.
• When the WfRun is created, a TaskRun is scheduled on the appropriate TaskDef.
• The RunWfRequest does not return to the user until that TaskRun is completed.
• If the TaskRun throws an error, the WfRun moves to ERROR and does not advance past the ENTRYPOINT Node. The error is returned to the client.
• If the TaskRun completes successfully, the WfRun advances past the ENTRYPOINT Node and the WfRun is returned to the client.

However, this introduces some additional complexities around timeouts, etc.

[diablouma]

@coltmcnealy-lh can provide more context

[coltmcnealy-lh]

@coltmcnealy-lh can provide more context

:joy: I thought you were going to write the ticket :smile: No worries, I'll edit the comment and put in a description.