Support for `relationship` construct in the language

[fractlrao]

The latest proposal is here - https://github.com/fractl-io/fractl/issues/482#issuecomment-1078736357

The current mechanism to associate an entity instance with another entity instance is a :ref. :ref has some constraints:

1. :ref is a uni-directional relationship - reference from the entity where it is defined to the :refed entity is possible, but not the other way round
2. :ref does not have a mechanism to specify the structure of the relationship. For example, we might want to specify that entity :Department contains instances of entity :Employee. Such structures are common in most domains and supporting them can be very useful for domain modeling
3. :ref captures a relationship. This relationship itself might have a lifecycle and many attributes (created date, modified date, current state - e.g., checkout/returned, checkout_on, returned_on, etc) - it would help to capture such attributes.
4. Finally, exposing actions that perform CRUD an association between two entities declaratively would be helpful. Currently :ref somewhat captures the relationship, but isn’t an independent construct to perform CRUD on. (example below)

Expanding the concept of :ref into a first-class construct in the language would be very helpful.

The proposal is to create a construct of a relationship - a special kind of record that captures the relationship between two entities:

(entity :Department …)

(entity :Employee …)

(relationship :DeptEmployee
  {:meta             {:type :contains
                           :cascading-deletes false
                           :cardinality 1-many}
   :Department  {:ref :Department.Id
                           :role :source
                           :refname :WorksFor}
   :Employee      {:ref :Employee.Id
                           :role :target
                           :refname :Workers}
   :StartDate     {:type :Kernel/DateTime}
   …})

The definition above should automatically create a :WorksFor attribute in :Employee and :Workers attribute in :Department.

A separate construct like above allows us to:

1. Relationships to be bi-directionally accessible
2. Model structure of the application (e.g., :type :contains implies cascading delete by default)
3. Capture information related to the relationship in a relevant construct (:StartDate)
4. Allow users to perform CRUD operation on this construct, similar to an entity.

[vijayfractl]

@fractlrao On point (2) - isn't this a case for composition? If what is meant is "multiple references to an entity" (and not "multiple instances of entity") a small extension to :ref should suffice:

(entity :Department
 {:Employees {:listof {:ref :Employee.Id}}
  ...})

If the relationship needs to be expressed as a separate structure, I think it can be achieved by :meta,

(entity :Department …)

(entity :Employee …)

(entity :DeptEmployee
  {:meta {:relationship [:Department :=> :Employee]}
   :Department {:ref :Department.Id}
   :Employee {:ref :Employee.Id}
   :StartDate {:default :Employee.JoinDate}})

I don't know how useful this kind of modelling is - because any data-query that can be done against :DeptEmployee will work equally well for a join on :Department and :Employee. Maybe this can act as a convenient "view" into such joins. Again, a sequence of employee-ids stored in :Department may have problems with indexing and reverse queries may not give the expected performance.

[fractlrao]

The relationship construct and the :meta approach are both aiming to achieve the same. Would we be able to express bi-directional relationship, cardinality and containment with the :meta approach and still keep it syntactically simple?

[vijayfractl]

It is possible to express bi-directional relationship, cardinality and containment without introducing a new construct to the language. But at this point we can continue to explore more examples/use-cases using the relationship syntax and try to define its full semantics, then think about simplifying the syntax.

[fractlrao]

It would help to add declarative constructs to the language. The concept of a dataflow, while being data-oriented, maps 1:1 to functions (except for idempotent operations). To make Fractl more declarative, bolstering features on the entity side of the language (e.g., relationships, :pre and :post invariants) would be helpful.

[vijayfractl]

@fractlrao Is an instance of relationship :DeptEmployee created automatically when an :Employee is added to the system? It seems the attributes :WorksFor and :Workers are persisted with :Department and :Employee. What about the :StartDate attribute?

[fractlrao]

Besides the containment use case, the following is a use case where a declarative syntax might be helpful.

In the context of the library example, users checkout and checkin books. It would help to capture this association between :User and :Book in a declarative fashion. With a relationship :Checkout between :User and ;Book, in place, a :User can perform CRUD operations on the relationship. Thereby, checking out (upsert on the relationship) and checking in (delete on the relationship) the book.

Such a declaration will allow the compiler to deduce that a “checkout” action is allowed "on a book by the user”. This information can be used in multiple places - for example, presenting a “checkout” button while displaying a book in the autogenerated UI.

For this to be complete, we will need to ensure that the user can declare :pre and :post checks/invariants (e.g., check whether the user has no pending fines (pre) and send out an email on checkout (post))

[fractlrao]

@vmatv8, I was thinking of the relationship as something that the user explicitly manages - creates, deletes instances of them, similar to entities.

[vijayfractl]

The relationship construct makes sense in the context of automatic UI generation. I think we need to define the possible or default UI layouts for each type of relationships. In other words, the visual representation for a 1-1, 1-N and N-N relationships. There could be other possible combinations as well - like 2-1, as in the case of accounts->transactions (i.e more specific cases of N-N).

While the approach is attractive, it brings up certain design and implementation challenges as well. Even for the same kind of relationships, the UI generated may have to be very different. Consider the employee-department relationship:

(relationship :EmployeeDepartment
 {:meta {:from :Employee :to :Department}
  ...)

The generated UI can just show a drop-down with all the department-ids in the employee form - a relationship can be established with the upserted-employee and the selected department.

A library-book checkout use-case has the same 1-N relationship between users and books:

(relationship :CheckoutBook
 {:meta {:from :User :to :Book}
  ...)

But the UI for establishing the relationship has to be much complex, probably with multiple screens - because facilities for searching for and listing the books has to be provided.

This means the relationship has to be combined with some kind of "layout templates" to be usable for automated UI generation. @fractlrao Please share your thoughts.

[vijayfractl]

Proposal for relationship construct - syntax and semantics

(entity
 :Library/User
 {:Name :Kernel/String
  :DOB :Kernel/DateTime
  :DateJoined :Kernel/DateTime})

(entity
 :Library/Book
 {:Title :Kernel/String
  :ISBN {:check isbn?}
  :Authors {:listof :Kernel/String}})

(relationship
 :Library/CheckoutBook
 {:User {:ref :Library/User.Id}
  :Book {:ref :Library/Book.Id}
  :Date :Kernel/DateTime
  :meta
  {:from :User
   :to :Book
   :cardinality {:type :1-many 
                 :exclusive true} ; a book can be checked-out at a time by a single user only
   }})

The :CheckoutBook relationship creates the following simple graph:

User----CheckoutBook--->Book

Usage scenario for the graph - automatic UI generation

From this graph, the UI for a dashboard can be auto-generated. This will contain a User input form because the entity :User is at the root of the graph. If there are multiple roots, input forms for each root entity will be laid-out on a grid.

Two actions can be performed from the input form - Create and Find. When a new instance is created or queried (by 'Find'), the input form will switch to a card view, with two possible actions - Edit and Cancel. Edit will display the input form with data loaded from the card. Cancel will return to the empty input view.

Any relationships that can be created from the entity will be available as additional actions (visually represented as buttons, links or menus). For example, the :User input form will have a Checkout Book action below it. Activating this will show an expanded view with a form for creating the relationship. This form will have the Create, Find and Find All actions. (Find All will display the results in a table, rather than a card). Additional actions for creating instances of the :to element of the relationship also will be available in this form. For instance, The Checkout Book form will have a Create New Book action. This will allow the user to navigate to a new input form which is auto-generated by treating the :to entity as the new graph root. (Once the UI drills into the graph, the generated forms will have an additional Back button which will allow the user to navigate up the hierarchy).

Relationship API

(fractl.component/relationships <component-name>)

Return the relationships graph for the component, structured as a map.
Example -

{:Library/User
 {:CheckoutBook :Book}}

(fractl.component/fetch-relationship-schema <rel-name>)

Return the full schema of the relationship.

(fractl.component/crud-event-listener <entity/rel-name> <callback-function>)

If any CRUD relevant to the entity or relationship happens within the graph, invoke the callback.
(The UI generation code can register such a callback and update relevant reagent-atoms
 for dynamically updating the view).

Let's look at one more example - where accounting transactions are modelled as a relationship.

(entity
 :Acme/Account
 {:Title :Kernel/String})

(relationship
 :Acme/Transaction
 {:Debit {:ref :Acme/Account.Id}
  :Credit {:ref :Acme/Account.Id}
  :Amount :Kernel/Decimal
  :Date :Kernel/DateTime
  :meta
  {:from :Debit
   :to :Credit
   :cardinality 
   {:type :1-1
    :exclusive false}}})

As the relationship has its :exclusive flag set to false, multiple instance of transactions with the same :Debit and :Credit accounts are possible. As a relationship prevents self-references, :Debit and :Credit cannot be the same account-id. No explicit constraint declarations are required for this.

The following cardinalities are supported in relationships: :1-1 (one-to-one), :1-M (one-to-many), :M-1 (many-to-one) and :M-M (many-to-many).

TODO: Provide additional example and usage scenarios