Implement ordering in observable:MessageThread

[ajnelson-nist]

Disclaimer

Participation by NIST in the creation of the documentation of mentioned software is not intended to imply a recommendation or endorsement by the National Institute of Standards and Technology, nor is it intended to imply that any specific software is necessarily the best available for the purpose.

Background

Issue 389 imports the Collections Ontology to provide UCO with a reusable representation for ordered lists.

co:List implements a strictly linear list - it can have at most one "beginning" member, and at most one "ending" member. It cannot fork.

Message threads in UCO, such as email reply threads, can link into a semi-ordered structure, and be non-linear via forking (two replies to the same message) or multiple In-Reply-to header values (possible per RFC 2822, Section 3.6.4). So, a co:List would not serve the needs of UCO's email-based message threads.

The design of co:List could be adapted into a new sibling class of co:List. This proposal does so, defining in the UCO Types namespace a class types:Thread that supports partial ordering using classes and properties from the Collections Ontology that co:List had subclassed. With this types:Thread class, observable:MessageThread is refined as a subclass of types:Thread, enabling representation of reply sequences.

Requirements

Requirement 1

UCO must have a class to represent partially ordered sets. This proposal will refer to that class as a Thread.

Requirement 2

A Thread must support one of its members being followed directly by 0, 1, or more other members.

Requirement 3

A Thread must support one of its members directly following 0, 1, or more other members.

Requirement 4

It must be possible to represent a gap in a thread's ordering, where some members are believed to follow others via an unknown but definite sequence of links.

Requirement 5

It must be possible to determine whether a member of a thread follows another member topologically, via knowledge of direct links or indirect links.

Requirement 6

observable:MessageThread must be able to represent a non-linear email thread, with 0, 1, or more In-Reply-to headers per email, and 0, 1, or more messages replying to any email.

Risk / Benefit analysis

Benefits

• The Thread class provides a substantially different behavior to the linear co:List class, and is capable of supporting complex email reply chains.

Risks

• The Thread class is based on an external dependency. Should something significant change in the Collections Ontology, the Thread class would also need to change.
• Other risks inherent to adoption of the Collections Ontology are detailed in Issue 389.
  • A risk to focus on is the interoperability conflict posed by UCO's MessageThreadFacet. What should become the subclass of Thread - MessageThread or MessageThreadFacet? This proposal chooses MessageThread because Facets, as a part of an object and not a "Whole" object, are likely to cause significant semantic interoperability concern if they were to become subclasses of other classes that represent "Whole" objects. There are valid arguments that a message being in a thread is domain-specific representation and inherent quality of the object, and thus would follow other UCO design examples and move MessageThreadFacet to being the subclass of Thread. However, to the proposer, this does not outweigh the interoperability concerns, nor the "Object Wholeness" matter.
• observable:message has to date represented the way to link a Message to a MessageThread, though without the ability to define an order. observable:message is a property housed in MessageThreadFacet, but has a similar structure to co:element which would reside on a Thread [sic.] object. This proposal (currently) makes no suggestions on changing the observable:message property.

Competencies demonstrated

Competency 1

Suppose an email message set is found, with In-Reply-to headers linking messages like this (where a higher integer connected to a lower integer means the message with the higher integer replied to the message with the lower integer):

 1     2     3
* --- * --- *
 \     \
  \     \ 4
   \     *
 5  \ 6
* --- *

 7
*

Competency Question 1.1

How many messages were replied to how many times, on a scale of 0, 1, multiple?

Result 1.1

.	.	.
Messages replied to ...	0 times	3, 4, 6
	1 time	5
	>1 times	1,2
Messages replying to ...	0 messages	1, 5
	1 message	2, 3, 4
	>1 messages	6
Messages not in thread		7

These are represented in the unit test tests/examples/message_thread_PASS.json, kb:message-thread-1 and kb:message-7.

Competency Question 1.2

What messages follow 1, topologically?

Result 1.2

2, 3, 4, and 6.

This query is encoded in a Python unit test in the PR, test_message_thread().

Solution suggestion

• Define a class types:Thread, analagous to co:List except instead of being a linear list, nodes within the list can follow multiple nodes and be followed by multiple nodes.
• Where the co:List has a designation for its first and last items, the Thread defines an "Origin item" as an item with no predecessor, and a "Terminal item" as an item with no successor.

The following table shows analogous classes and properties. Each property is cited with rdfs:seeAlso, and each class is designated owl:disjointWith, in the attached PR.

co:List	types:Thread
co:ListItem	types:ThreadItem
co:firstItem	types:threadOriginItem
co:followedBy	types:threadSuccessor
co:lastItem	types:threadTerminalItem
co:nextItem	types:threadNextItem
co:precededBy	types:threadPredecessor
co:previousItem	types:threadPreviousItem

• Define SHACL shapes, favoring the standalone property shape applied with sh:targetSubjectsOf as with the Collections Ontology issue.
• Implement tests to exercise the Thread class as the co:List was exercised in the Collections Ontology proposal.
• Implement a sample and tests for the illustration in Competency Question 1.

In support of OWL inferencing, and to maintain parallel structure with the Collections Ontology, some OWL constructs are ported from the co:List class. The ported constructs are cited with rdfs:comments and/or rdfs:seeAlso.

Coordination

• Tracking in Jira ticket OC-88
• [x] Administrative review completed, proposal announced to Ontology Committees (OCs) on 2022-05-26
• [x] Requirements to be discussed in OC meeting, 2022-06-30
• [x] Requirements Review vote occurred, passing, on 2022-06-30
• [x] Requirements development phase completed.
• [x] Solution announced to OCs on 2022-06-30
• [x] Solutions Approval to be discussed in OC meeting, 2022-07-12
• [x] Solutions Approval vote occurred, passing, on 2022-07-12
• [x] Solutions development phase completed.
• [x] Implementation merged into develop
• [x] Milestone linked
• [x] Documentation logged in pending release page

[sbarnum]

I would disagree with the proposal to make observable:MessageThread a subclass of the newly proposed types:Thread as it is likely to have all the complex semantic entailment issues as outlined in Risk 7 of CP-389 (https://github.com/ucoProject/UCO/issues/389). observable:MessageThread MUST remain a subclass of observable:ObservableObject. If it is also made a subclass of types:Thread that is likely to cause issues. It should also not be made a subclass of observable:MessageThreadFacet for similar reasons.

I would propose that a much more straightforward and low risk approach would be to not alter the subclassing of the observable classes at all and rather simply rename the observable:message property to observable:messageThread and change its range to be types:Thread instead of observable:ObservableObject. This enables the full expression of the ordered message thread, aligns to the current intended meaning of the observable:message property, keeps consistent with all current semantics for ObservableObjects and for observable object Facets, and avoids the semantic entailment complexities of hybrid subclassing between UCO-native classes and CO classes.

[ajnelson-nist]

... rename the observable:message property to observable:messageThread and change its range to be types:Thread instead of observable:ObservableObject. ...

@sbarnum , because I remembered UCO's usage of types:ControlledDictionary, I will follow that referential pattern, modeling on observable:exifData.

Also, I had forgotten that UCO does not have observable:messageThread as a property. I'd misremembered an old stand-in property from the messages.json example.

I'll adjust this proposal to add observable:messageThread. We can decide on the call today whether observable:message should be deleted, in light of an unordered attachment pattern now being available via the property path observable:messageThread / co:element.

[ajnelson-nist]

I'm further inclined to recommend observable:message be deleted because it has a SHACL constraint requiring at least 1 object be referenced.

This is another instance of what I have found to be a harmful accident from the OWL translation. An OWL minimum cardinality 1 expresses that there exists at least one message for any given message thread. I agree there. The SHACL constraint fails your data if you don't make an explicit reference to a message, which is harmful because you might want to express that a thread exists with a concrete identifier, but you might not know yet which message objects are its members.

So I will recommend to the committee we also delete observable:message.

[ajnelson-nist]

Solution vote today incorporates that observable:message should be removed from observable:MessageFacet, and also removed from the ontology.