Make refs bi-directional

[vijayfractl]

Consider the following model:

(entity :Acme/Department
 {:Name :Kernel/String})

(entity :Acme/Employee
 {:FirstName :Kernel/String
  :LastName :Kernel/String
  :Dept {:ref :Acme/Department.Id}})

:Employee has a :ref to :Department - given an employee it's now possible to navigate to the related department. The reverse is not possible - given a department an explicit user-defined dataflow is required to query all employees in it. This can be automated if a bi-directional switch is enabled for :refs.

(entity :Acme/Employee
 {:FirstName :Kernel/String
  :LastName :Kernel/String
  :Dept {:ref :Acme/Department.Id
         :two-way true}})

The :two-way flag will allow the compiler to emit the required code for other parts of the system (like the UI-generator) to efficiently do the reverse lookup.

[vijayfractl]

Two-way refs could be expressed as a more generic relationship construct:

(entity :Acme/Department
 {:Name :Kernel/String
  :meta {:contains :Acme/Employee}})

(entity :Acme/Employee
 {:FirstName :Kernel/String
  :LastName :Kernel/String})

(relationship :Acme/EmployeeAssignment
 {:Emp :Acme/Employee
  :Dept :Acme/Department
  {:meta {:unique true}})

This will establish a unique reference from an employee to a department. The compiler will emit code to ensure that the referenced department exists and for doing other validity checks. A relationship is by default bi-directional (unless the :two-way flag is turned-off). In other words, relationships build bi-directional graphs.

The :contains meta of :Department indicates that a relationship from :Employee is expected, if such a relationship is not defined the compiler will issue a warning. This meta can have other uses as well, e.g for UI generators for creating embedded employee lists for department dashboards.

The following example shows how a new relationship could be established:

(dataflow :Acme/AssignEmployee
 {:Acme/EmployeeAssignment
  {:Emp :Acme/AssignEmployee.EmpId
   :Dept :Acme/AssignEmployee.DeptId}})

When a new relationship is created, the runtime makes the following checks:

1. :Emp and :Dept are valid, existing Ids of the appropriate entity-instances
2. the uniqueness of the relationship is not violated, i.e if a relationship instance with the same employee and department Ids already exist - an error is raised

The runtime will manage the graph-data transparent to the application - it may use the default store or a dedicated graph database for this purpose.

Let's look at another example of relationships, this time in the context of a double-entry accounting application:

(entity :Acme/Account
 {:Title :Kernel/String})

(record :Acme/TransactionDetail
 {:Date {:type :Kernel/DateTime :default now}
  :Amount :Kernel/Decimal})

(relationship :Acme/Transaction
 {:Debit :Acme/Account
  :Credit :Acme/Account
  :Detail :Acme/TransactionDetail
  {:meta {:data :Detail
          :unique false
          :exclusive true}})

Here a transaction is modelled as a relationship between two accounts. The same combination of accounts can appear in multiple transactions, so the :unique flag is set to false. An account cannot refer to itself in a transaction on both the debit and credit sides - so the :exclusive flag is turned on. The graph edge also need to carry some additional data which pertain to the details of the transaction.

(dataflow :Acme/NewTransaction
 {:Acme/TransactionDetail
  {:Amount :Acme/NewTransaction.Amt}}
 {:Acme/Transaction
  {:Debit :Acme/NewTransaction.Debit
   :Credit :Acme/NewTransaction.Credit
   :Data :Acme/TransactionDetail}})

Cardinality

The cardinality of a relationship can almost always be inferred from its other properties. For instance the unique flag indicates a 1-1 relationship, when it's turned-off the cardinality will be N-N.

More specific cardinality can be expressed as shown in the following examples:

;; A single instance of :E1 can connect to n :E2s
(relationship :R1
 {:X :E1
  :Y :E2
  {:meta {:cardinality {:X 1 :Y :N}}})

;; A single instance of :E1 can connect to exactly 3 :E2s.
;; An attempt to create the fourth edge will result in an error
(relationship :R2
 {:X :E1
  :Y :E2
  {:meta {:cardinality {:X 1 :Y 3}}})

Attributes of a relationship

A relationship should always have two user-defined attributes that will form the two nodes on the graph. An additional attribute may be defined, but it must be marked as :data in the meta section, failing to do this will raise an error.

A relationship instance will be assigned an autogenerated :Id attribute, which can be used for deleting it (using the :delete command).

This means a properly formed relationship instance may have a maximum of four attributes.

The :meta section of a relationship can define the following properties:

:unique - boolean (default true)
:exclusive - boolean (default false)
:data - path (keyword or string, optional)
:cardinality - vector (default value will be inferred from :unique and :exclusive flags)
:two-way - boolean (default true)

[fractlrao]

@vmatv8, thank you for the detailed proposal. I have a few minor comments:

1. Can :contains be moved to the relationship :meta? (:contains [:Dept :Emp] of just :contains :Emp)
2. Would it be possible to define the ref from the relationship to the entity using any unique attribute of the entity, not just :Id? Reusing the existing :ref definition might serve us well here (:Emp {:ref :Acme/Employee.Id} or :Emp {:ref :Acme/Employee.Username})
3. To keep the concept of relationships simple, it might help to defer introducing the concept of relationship as a record i.e., carrying data (the :Transaction use case above) and limit it to defining structure for the application.

[fractlrao]

After this relationship has been created, a dataflow pattern might need to access the :Department of an :Employee?

(dataflow :NotifyEmployee
  {:Employee {:Username? :NotifyEmployee.Username}}
  {:Twilio.SMS/Message {:Body '(str :Employee.Name, " Your new hours are: " :Employee.Dept.Hours)}})

The path in the above example (:Employee.Dept.Hours) references :Dept - would this work automatically because :Dept is defined in the relationship? Or would it be :Employee.EmployeeAssignment.Dept.Hours?