Pattern for CRUD gateway and relational records

[pospi]

There has been some discussion about what's best for managing these kinds of "relationship" records, and how to make things more ergonomic for the caller. We have particularly focused on the logic for update behaviours (adding, removing and editing relationship records), and on "shorthand" data structures for defining dependant records (eg. declaring fulfillment relationships when writing an economicEvent).

After thinking about this for a while my opinion is that we should not bother with these kinds of indirect edit methods and simply treat "relationship" records (like fulfillment) as first-class items with their own CREATE, UPDATE & DELETE methods. The reasoning is as follows:

Create:

There is a small efficiency gain to the caller if they can specify a sub-structure for relationship records when creating a parent. The internal logic for implementation is also deemed to be reasonably simple, as it is simply the composition of the creation of the base record and creation of any child relationship record(s). However, the sub-records for declaring relationships would necessarily differ from the stored records themselves (for example, the fulfilled_by field of the fulfillment would come from the newly created economicEvent). And, in practise, this logic and field munging ends up being cumbersome and somewhat of a burden to implementors. With the goal of making development of HoloREA as accessible as possible, I think we should simply omit this burden. All calls are going through a websocket at the end of the day anyway.

Update & Delete:

Any set-based implementations for UPDATE & DELETE of referenced records within the api involve complex logic for retrieving the affected record and / or managing the change. However, it is pointless for the backend to be forced to run these computations and checks! Any UPDATE or DELETE request is going to originate at the UI layer, by the action of a user who has clicked on some control next to a data item. In all cases, the ID of the affected relationship is already known by the UI at time of operation. There is no need to say "remove this ID from the set of an economicEvent's fulfills", we only need to say "delete the fulfillment with this ID". And the same applies to UPDATE.

Delete:

Perhaps worth forking to a separate issue if there is contention, but specific to DELETE is the way that we manage the deletion- do we remove links, or records? I think this will be a case-by-case thing, but expect that in most cases we would want to remove the record rather than the link. This allows UIs to load previous versions of records where they have been deleted, by following the dangling link pointer back to the previously available version of the referenced record.

Either way it seems as though the decision to make historical information easily visible or not will fall to the UI layer, and the particular norms and requirements of the context in which a consuming application is being built.

[fosterlynn]

my opinion is that we should not bother with these kinds of indirect edit methods and simply treat "relationship" records (like fulfillment) as first-class items with their own CREATE, UPDATE & DELETE methods.

I agree. For another thing, sometimes they will actually be M:M relationships, which adds even more to the complexity.

There is a small efficiency gain to the caller if they can specify a sub-structure for relationship records when creating a parent.

Do you mean by parent the Commitment in Fulfillment and Intent in Satisfaction? If so, then I very much agree to not creating any Fulfillment or Satisfaction ahead of time, we should never assume that something will be fulfilled or satisfied, and we won't know how many either.

specific to DELETE is the way that we manage the deletion- do we remove links, or records? I think this will be a case-by-case thing, but expect that in most cases we would want to remove the record rather than the link.

I can't imagine any case we would not want to remove the record itself (or mark it deleted if that is how it works), we can't assume access through the links. But I may not understand you, if not, apologies.

[pospi]

All good, I think we are on the same page. Just to clarify a couple of these things while we're here:

I very much agree to not creating any Fulfillment or Satisfaction ahead of time, we should never assume that something will be fulfilled or satisfied, and we won't know how many either.

Yes. But ditto to not providing for the facility to manually create any Fulfillments or Satisfactions within the same operation as an Event / Commitment. In those cases the caller would know ahead of time, but their ability to specify all this in one statement rather than with multiple commands isn't worth the additional complexity.

any case we would not want to remove the record itself

So this might be a few different patterns:

• for indirect references (eg. Satisfaction & Fulfillment) there are actually 5 different Holochain primitives involved- the "middle" record itself, and bidirectional links at each "edge" (ie. EconomicEvent.fulfills & Fulfillment.fulfilledBy).
• for direct references (eg. Claim.triggeredBy) there are just the two components of the link and its reciprocal.

Thinking about it, in all cases it might be best to delete everything (all 5 primitives in indirect references, both in direct references). Otherwise determining the linked addresses for a field requires 2 hops per field (follow link to entry to check deletion flag) rather than 1 (simply fetching all links and ignoring those marked deleted).

That said, some way of retrieving deleted references of both types needs to be provided; failing to do so is simply creating an unequal power dynamic in technical vs. non-technical users of the system. Still maintain that the decision to make archival information visible is that of the UI application.

[pospi]

I think there is enough certainty here now to proceed with #42.