rules as a language feature

[vijayfractl]

Rules

Note This new feature will deprecate conditional-dataflows (#185).

A new language construct that will enable the Fractl evaluator to function as a "rules engine". This construct will allow conditional-patterns to be evaluated after create/update/delete of instances. If the conditions evaluate to a truth value, some consequent patterns are evaluated. An example is shown below:

(component :Acme.Core)

(entity :A {:Id :Identity :X :Int :Y :Int})
(entity :B {:Id :Identity :R :Int})
(entity :C {:Id :Identity :S :Int})
(event :E {:Message :String})
(dataflow :E [:eval '(println :E.Message)])

(rule :OnZeroX
 {:A {:X 0}}
 :then 
 {:E {:Message '(str :A.Id " is now zero")}})

The rule construct has the following syntax:

(rule <rule-name> <conditional-patterns> :then <consequent-patterns> 
                  {:meta {:priority <number> :passive <boolean> :category <category-name>}})

For the :OnZeroX rule, there's just one conditional-pattern - {:A {:X 0}}, and one consequent pattern - {:E {:Message '(str :A.Id " is now zero")}}. (There can be more than one pattern in the conditional and consequent sections). Whenever an instance of :A is created or updated in the system (as part of evaluating a dataflow), the conditional-pattern is applied to that instance. The pattern will return true if the instance's :X attribute is 0 and this in-turn will cause the consequent pattern to evaluate, resulting in the printing of a message to the console. (If the consequent is an expression of the form (f arg1 arg2 ...), the expression will be evaluated to get a pattern and then that pattern is used as the consequent).

Note that the evaluation of the rule will not block the thread that triggered it, i.e. the thread running the dataflow that performed the CRUD on the :A instance. Rules are evaluated asynchronously and their failures are tracked separately by the runtime. A rule that continues to be in failure (after a threshold has reached) will cause the CRUD operations themselves to fail.

Multiple rules could be defined on the same entity and they will be evaluated in an arbitrary order. For example, both the following rules will be executed when the instance {:A {:X 0 :Y 100}} is created in the system, in which order they are executed is decided by the runtime.

(rule :R1
 {:A {:X 0}}
 :then
 {:E1 { ....}}
 )

(rule :R2
 {:A {:Y 100}}
 :then
 {:E2 { ....}} 
 )

The order of execution can be overridden by the user with the :priority option:

(rule :R1
 {:A {:X 0}}
 :then
 {:E1 { .... }}
 {:meta {:priority 1}}
 )

(rule :R2
 {:A {:Y 100}}
 :then
 {:E2 { .... }}
 {:meta {:priority 2}}
 )

The rules with the highest to the lowest priorities are selected for execution, in that order. In this case :R2 will be executed first and :R1 next. Also note that the rules selected for execution for an instance will run sequentially (in a thread separate from the triggering-dataflow's thread, as noted above).

Multiple conditional-patterns

The following rule is defined to listen for changes on both :A and :B -

(rule :R3
 {:A {:X 100}}
 {:B {:R 200}}
 :then
 {:C {:S '(+ :A.Y :B.R)}})

The rule :R3 is triggered when either an instance of :A or :B is upserted in the system. The conditional-patterns will succeed only if instances of both :A and :B is available in the dataflow-environment that triggered the rule.

This feature in covered in detail in the issue #1254.

Rules with contains-relationships

The following rule will be triggered when a high-priority support-ticket is created under a "primary" customer:

(entity :Customer ...)
(entity :SupportTicket ...)
(relationship :CustomerSupportTicket
 {:meta {:contains [:Customer :SupportTicket]}})

(rule :HandlePrimaryTicket
 {:SupportTicket {:Priority "high"}
  :-> [[:CustomerSupportTicket {:Customer {:Type "primary"}}]]
  :as [:ST]}
 :then
 {:AssignSupportExecutive {:SupportTicketId :ST.Id :CustomerId :Customer.Id}})

This feature is covered by issue #1255.

On-delete rules

The :delete clause can be used to mark a rule for execution when an instance is deleted. Example:

(rule :R4
 [:delete {:A {:X 0}}]
 :then {:E {:Message '(str "deleted - " :A.Id)}})

Conditional-pattern unification

The conditional-patterns in a rule has a syntax similar to dataflow patterns, but they are not evaluated by the core Fractl interpreter. Instead they are "unified" with an instance. The unification is applied using the following simple rule:

The value of an attribute in the pattern will be matched with the corresponding value in the instance -
if all the values match, the unification succeeds. (As in other rule-engines, unification do not result in variable bindings).

A rule that involves a contains-relationship may trigger an internal query-operation and apply the unification on the result. This will happen transparent to the user.

The value part of an attribute may be a conditional expression using one of the comparison operators: :=, :<, :<=, :>, :>=, :<>, :in, :between. They work for numeric, string and date-time types. Comparison expressions maybe combined together using the logical operators :or and :and. Some examples:

(rule :R5
 {:A {:X [:<= 5]}}
 :then
 ;; do something
 )

(rule :R6
 {:A {:X [:or [:= 100] [:<= 5]]}
 :then
 ;; do something
 )

(rule :R7
 {:A {:X [:between 10 20]}}
 :then
 ;; do something
 )

(rule :R8
 {:A {:X [:in 1 9 20 100]}}
 :then
 ;; do something
 )

There's also an operator specifically for strings called :like that checks whether the value starts with a specific prefix.

The attribute-value could also be compared with the help of a predicate:

(defn x-for-r6? [x]
  (or (= x 100) (<= x 5)))

(rule :R6
 {:A {:X x-for-r6?}}
 :then
 ;; do something
 )

Decisions

A new dataflow pattern is proposed to explicitly evaluate rules with a fallback mechanism to LLM. Syntax of the construct:

[:decide <rule-category-or-name> <input-instances> :check <output-type> :as <alias>]

An example is shown below:

(entity :Order { ... })
(entity :DeliveryMode { ... })

(defn make-del-mode [type order]
  {:DeliveryMode {:Type type}
  :-> [[{:OrderDeliveryMode {:Order (:Id order)}}]]})

;; decide mode-of-delivery for the order
(rule :R1
 {:Order {:TotalPrice [:>= 5000]} :as :O}
 :then '(make-del-mode "fast" :O)
 {:meta {:passive true :category :DeliveryRules}})

(rule :R2
 {:Order {:TotalPrice [:< 5000]} :as :O}
 :then '(make-del-mode "normal" :O)
 {:meta {:passive true :category :DeliveryRules}})

(dataflow :CreateOrder
 {:Order { ... } :as :O}
 [:decide :DeliveryRules [:O] :check :OrderDeliveryMode :as :DM]
 {:Result {:Order :O :Mode :DM}})

Note that the two rules defined in the example are declared as :passive, which means they have to be explicitly invoked by a :decide call. The :decide call will look for all passive rules defined in the :DeliveryRules category and execute those rules. If no such rules are defined, :decide will try to invoke an LLM to get a :DeliveryMode. The LLM is picked from an inference as defined below:

(inference :Acme.Core/UserSupport
 {:category :DeliveryRules
  :seed [{:user "show the delivery-mode for {:Order {:TotalPrice 4500}}"
           :assistant "{:DeliveryMode {:Type \"normal\"}}"
          ; ...
         ]
  :embed [:Order :DeliveryMode]})

(The syntax and semantics of inference will be covered in a separate issue).

If no inference entry is found for an LLM in the :DeliveryRules category, the dataflow will fail with an error.

The implementation of the :decide construct will be tracked in #1256.

[fractlrao]

Some minor comments:

1. Instead of [:on-delete], can we use the delete dataflow pattern:

   (rule :OnZeroX
   [:delete {:A {:X 0}}]
   :then 
   {:E {:Message '(str :A.Id " is now deleted")}})

2. Can we support multiple patterns without a vector (both before and after :then similar to dataflow syntax)?

   (rule :R3
   {:A {:X 100}}
   {:B {:R 200}}
   :then
   {:C {:S '(+ :A.Y :B.R)}})

3. For :[decide ...], the current syntax might be ambiguous if there are multiple types of rules for a given entity. For example, there could be multiple kinds of rules for orders (delivery mode, whether approval is required, etc). It might help to specify what "class of rules" need to be invoked [:decide :DeliveryRules [:O] :as :DM :check :DeliveryMode]

(proposed syntax for :decide: [:decide <rule-category> <list of entity inputs> :check <expected-result-entity> :as <alias>])

Rules need to carry some information about the category. For example in their names as below:

(rule :DeliveryRules.R1
 {:Order {:TotalPrice [:>= 5000]} :as :O}
 :then '(make-del-mode "fast" :O)
 :dormant)

(rule :DeliveryRules.R2
 {:Order {:TotalPrice [:< 5000]} :as :O}
 :then '(make-del-mode "normal" :O)
 :dormant)

[vijayfractl]

Issue description updated to take care of the above enhancements suggested by @fractlrao.