Introduction

This PR implements a major update to the ZEN Engine. It introduces powerful new capabilities, including the addition of user-defined graph nodes and introduction of ZEN templating language, making it more versatile and customisable than ever before.

User-Defined Graph Nodes (Custom Nodes)

One of the most requested features from the community was a way for them to define their own graph nodes, referred to as custom nodes. This enhancement significantly increases the flexibility and customisation options available within the ZEN Engine, allowing users to tailor the engine to their specific needs and workflows.

Features:

Customization: Users can now create nodes that perform specific actions or calculations, tailored to their unique business rules and logic.
Flexibility: The addition of custom nodes introduces a new level of adaptability, enabling the engine to handle a wide range of scenarios and use cases.

API

Below is a short example illustrating usage of Custom Nodes within Node.js ecosystem.

const engine = new ZenEngine({
  loader,
  customHandler: async (request) => {
    const left = request.getField('left') as number;
    const right = request.getField('right') as number;
    const key = request.getFieldRaw('key') as string;

    return {
      output: {[key]: left + right}, 
      trace: { myTraceData: { left, right, key } }
    };
  }
});

const r = await engine.evaluate('custom.json', {a: 5});
expect(r.result.data).toEqual(25);

Largely APIs through all languages will follow mostly similar patterns (below is TypeScript definition of the custom node handler).

type CustomNodeHandler = (request: ZenEngineHandlerRequest) => Promise<ZenEngineHandlerResponse>;

export type DecisionNode = {
  id: string;
  name: string;
  kind: string;
  config: any;
}

type ZenEngineHandlerRequest = {
  input: any;
  node: DecisionNode;
  getField(path: string): unknown; // Invokes ZEN Template under the hood
  getFieldRaw(path: string): unknown; // Gets the field as raw property
}

type ZenEngineHandlerResponse = {
  output: any;
  traceData?: any;
}

ZEN Templating Language

Alongside the custom nodes, we've also introduced the ZEN templating language. Powered by ZEN Expressions, the templating language supports full standard ZEN Expressions.

For example:

some string {{ 1 + 1 + a.nested }} // returns string
{{ my.prop + 10 }} // returns number

It can be invoked in all languages using (example TypeScript):

const data = await renderTemplate('{{ a + 10 }}', { a: 5 }); // Number(15)
const data = await renderTemplate('Hi {{ firstName }} {{ lastName }}', { firstName: 'John', lastName: 'Doe' }); // String('Hi John Doe')

FFI Performance

For a simple custom node as described above the performance is all languages is generally below 100 micro seconds (tested on Macbook M1). Namely:

Node.js - around 15µs - 20µs
Python - around 40µs - 60µs
Go - around 6µs - 12µs

Exposing Custom Nodes in JDM Editor

See: https://github.com/gorules/jdm-editor/pull/30. (WIP)

Note

The API is still subject to change and will be after the release as we monitor and start organising and preparing for V1 milestone.

gorules / zen

feat: custom node #138