Allow Passing `NodeOutput` into Task

[coltmcnealy-lh]

Right now, the following is illegal:

NodeOutput itemPrice = wf.execute("fetch-price", ....);

wf.execute("charge-customer", itemPrice);

You have to:

WfRunVariable itemPriceVar = wf.addVariable("item-price", INT);
wf.mutate(itemPriceVar, ASSIGN, wf.execute("fetch-price", ...);

wf.execute("charge-customer", itemPriceVar);

We should be able to do the following:

wf.execute("charge-customer", wf.execute("fetch-price", ...));

Implementation

This can be implemented on the client side. It should:

1. Create an internal variable transparently.
2. Assign the node output to that variable.
3. Transparently use that variable.

This does not require changing the protobuf.

Rejected Alternative

A rejected alternative is to change the protobuf to allow referring to a NodeRun. The problem with this is:

• With NodeRetentionPolicy and ThreadRetentionPolicy, we may refer to a NodeRun that has been garbage-collected.
• Loops get tricky: there could be thousands of NodeRuns for a single node.
• The developer experience is equal in both cases (perhaps better in my proposed case since the variable is visible on the dashboard).

[coltmcnealy-lh]

I started the implementation on this commit here: https://github.com/littlehorse-enterprises/littlehorse/commit/7517105b7c39d34d4cb254c8a385528a37fa226d

However, there are problems:

• The design involves declaring a transparent variable to hold the NodeOutput
• All variables must have a type
• We currently have no way of knowing the type returned by a given NodeRun

Solution: Require Client in Workflow

We now have output type information on a TaskDef and UserTaskDef. We can add it for an ExternalEventDef, which would mean that (with LH API access) we would be able to infer the output type of all nodes.

If the WfSpec builder SDK had access to a GRPC client, then we could simply fetch the required metadata from the server in order to compile it.

Advantages

We plan to enable better stacktraces and error handling by validating in-line during the compilation rather than just letting the LH Server return an error message with only node names. The only way to do that is to make the SDK call the LH API during node definition time (eg. WorkflowThread#execute()) and validate each step there. This would allow users to have a stacktrace from the specific line that caused a problem.

This means that we will have use-case for making the Workflow object require a client as well.

Disadvantages

• Making the Workflow object take in a client would be a breaking change.
• Some users may want to be able to just create a WfSpec without access to LH, save it to a file on disk, and deploy it from a pipeline. If the Workflow object requires client access, then they will not be able to fit this development flow.

Solution: Optionally Take Client

In this solution:

• We add a new constructor for the Workflow class that takes in an LHConfig
• If the config/client is provided, then we do validation as described above.
• If no config/client is provided, then we throw an IllegalStateException saying "you cannot directly pass a NodeOutput into another task without providing a client as we cannot infer the types."