Transclusion

In the document "Preface to the 1993 Edition" T. Nelson says "« Transclusion » is the term I use now, a word coined since the book was originally written. Transclusion will be a fundamental service of tomorrow's literary computers, and a property of the documents they will suply. Transclusion means that part of a document may be in several places -- in other documents beside the original -- without actually being copied there. This book is concerned in large part with the power this new kind of writing- with transclusive quotation - will bring". Then he adds up "In the body of this book, written before I thought of the word « transclusion », I simply use the term « inclusion ». I have regretfully left that as is. The concept is much better carried by the word « transclusion », so try to say it under your breath when you see the other term".

There are 2 points I find really interesting :

A transcluded part of a document, or a document, may resides in other documents without being copied,
The use of the term « inclusion » is related to the concept of « transclusion ».

Originally called « inclusion »

In "Literary Machine, Chapter 4" T. Nelson emphasis that a quote-link is not an inclusion and he says "While a front end may readily ask which portions of a document are inclusions, inclusions are not ordinarily flagged or shown unless a user requests it." "... the quote-link is essentially an annotation of an inclusion".

Link and transclusion

I understand that a quote-link is different from an inclusion because a quote-link would be part of the document which has this link while an inclusion wouldn't be a part of the document because transcluded content is not copied. Moreover the front-end must show the original document which the quote-link refers to, but a transclusion is a part of a document "readily ask" in order to be included without the need to show the user what has been transcluded.

Different types of transclusion

T. Nelson defines 2 differents transclusions called « transdelivery » and « transquotation ».

Transdelivery

That would be what I think of in the first place when I hear « transclusion ». A part of a document or a whole document showed in another document which transcluded this part or the whole document. What is transcluded doesn't reside in the document who contains the transclusion, thus to show what is transcluded the front-end has the task to "ask" the docuverse what is/are the transcluded content(s) and then to deliver the document which contains the transclusion. As a result the front-end show a loaded document which contains transclusion(s).

Transquotation

This is where it becomes tricky to me. I find it tough to envision transquotation without thinking of quote-links. In Xanadu's website transquotation is defined as a "quotation which remains connected to its origins." There one way I can comprehend this definition and which doesn't conflict with the definition of quote-link, but I find it to be bit problematic. A transquotation is simply a quotation which is not copied in the document. Now what I find delicate is when we think of « transquotation » and « transdelivery », they kinda are the same. To solve this problem, should we consider that a « transdelivery » doesn't remain connected to its origins? In case it does, why would a « transdelivery » not be a « transquotation » and vice versa?

Conclusion

A transclusion is a content which is a portion of a document or an entire document included in another document. The content is not copied, which means that the document which includes the content doesn't store the content in raw, despite this the included content is retrieved from the origin document in order to be shown in the document which transcludes the content. Lastly we have to strictly define what is the real nature of a « transdelivery » and a « transquotation ».

An idea toward a Map/Reduce solution for transclusion

Since we didn't implement the tumbler span envisioned by T. Nelson we will have to work around this method.

What is the aim

We want to let the front-end (or whatever which loads a document) to merge transcluded content(s) in order to build a full requested document.

String parsing

The parsing would be the role of the mapping. When requesting a document a map function look at the document content searching special spans, lets say we have two documents where their id is respectively A and B given by :

Document A

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed arcu libero, ullamcorper sed elementum ac, semper ac tortor. Integer ac elit turpis. In hac habitasse platea dictumst. Duis ornare congue libero eu convallis. Pellentesque quis libero molestie, imperdiet tellus eu, posuere lacus. Morbi tristique commodo feugiat. Nam egestas magna eu egestas tincidunt. Proin in blandit ligula. Praesent auctor commodo dui a dictum. Vivamus id consectetur eros.

Integer non lorem a ligula egestas condimentum ut faucibus lectus. Nam ultricies id metus a tristique. Donec vel justo erat. Mauris eget eros nulla.

Document B

In sodales est eget pulvinar semper. Sed nec purus eu turpis consequat aliquet. Aenean laoreet nisl eget arcu dictum, sit amet ultricies urna fringilla. Phasellus sapien quam, suscipit quis rutrum eu, rhoncus non nunc. Proin erat neque, placerat vitae ligula ac, blandit tempor mauris.

Now we want to transclude Document B within Document A :

Document A < B

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed arcu libero, ullamcorper sed elementum ac, semper ac tortor. Integer ac elit turpis. In hac habitasse platea dictumst. Duis ornare congue libero eu convallis. Pellentesque quis libero molestie, imperdiet tellus eu, posuere lacus. Morbi tristique commodo feugiat. Nam egestas magna eu egestas tincidunt. Proin in blandit ligula. Praesent auctor commodo dui a dictum. Vivamus id consectetur eros.

{{ B }}

Integer non lorem a ligula egestas condimentum ut faucibus lectus. Nam ultricies id metus a tristique. Donec vel justo erat. Mauris eget eros nulla.

Spans can be identified thanks to the double brackets "{{ refer to }}".

Reduce the string parsing

Once the content has been parsed in order to find what it's been transcluded, a simple reduce function will merge the content refered by the transclusion (take it as whats inside the double brackets) with the content of the requested document.

Issues

We now have to resolve what would be the difference between « transdelivery » and « transquotation ». If we consider a tranquote to refer to the source we can modify a bit the format of the double brackets by adding for instance "{{ =B }}" meaning that the reduce function should consider the reference to B while reducing,
This solution only allows to transclude documents in their entirety. I can't figure out how we could transclude portions without the tumbler concept.

Here is a solution : ee90697a7e2b83a10fdd6459e41afa83536f1681

No reduce

Firstly I want to precise that I didn't use any reduce function. The way I saw things with a reduce function is that I would have been able to get transcluded document's data thanks to include_docs=true but it appears that is not faisable.

View collation

Instead of reduce function I chose to setup a view collation in order to sort the documents by the parent and the transcluded documents.

List to render transcluded documents

A list function will then merge all documents and transcluded documents. The actual solution doesn't store the reference, it's a transdelivery. To implement a transquote we can easily adapt the list function to return referenced documents thanks to the symbole = in {{ =B }}.

Kown issue

Lets say A transcludes B and B transcludes C. When the list merge the documents, it wont take C into account when merging B to A. I'm working on this issue.
As I previously said, we can only transclude entire documents.

bffc9d747d36584606585c7a64d7ab221c70b684

Transclusion syntax

I decided to change the syntax in order to see what is transcluded and how the frontend should show it (transdelivery or tranquotation). Lets see this with an example.

Sample

Document A

{
  "data": "Document A trancludeliver B {{ B }} and transquotes C {{ =C }}"
}

Document B

{
  "data": "Document B transquote C {{ =C }} : Document B"
}

Document C

{
  "data": "It's Document C"
}

Document D

{
  "data": "{{ =B }}"
}

Results

GET docuverse/transclude?include_docs=true

[
  {
    "_id": "A",
    "_rev": "2-cb26663b98ecb7d512eb179e635d72e3",
    "data": "Document A trancludeliver B [Document B transquote C [It's Document C](<C>) : Document B](B) and transquotes C [It's Document C](<C>)"
  },
  {
    "_id": "B",
    "_rev": "1-7c7b2f35c140ea8997017e079c94b55d",
    "data": "Document B transquote C [It's Document C](<C>) : Document B"
  },
  {
    "_id": "C",
    "_rev": "1-46070e234a0bdd009c40f31aef04b62e",
    "data": "It's Document C"
  },
  {
    "_id": "D",
    "_rev": "1-d5a38812f7d05f16334e6b99d90f74cc",
    "data": "[Document B transquote C [It's Document C](<C>) : Document B](<B>)"
  }
]

GET docuverse/transclude?include_docs=true&startkey=["A"]&endkey=["A",{}]

[
  {
    "_id": "A",
    "_rev": "2-cb26663b98ecb7d512eb179e635d72e3",
    "data": "Document A trancludeliver B [Document B transquote C [It's Document C](<C>) : Document B](B) and transquotes C [It's Document C](<C>)"
  }
]

Format

[data](document id) : is a transdelivery

[data](<document id>) : is a transquote

benel / XanaduReloaded

Transclusions #2

Transclusion

Originally called « inclusion »

Link and transclusion

Different types of transclusion

Transdelivery

Transquotation

Conclusion

An idea toward a Map/Reduce solution for transclusion

What is the aim

String parsing

Document A

Document B

Document A < B

Reduce the string parsing

Issues

No reduce

View collation

List to render transcluded documents

Kown issue

Transclusion syntax

Sample

Results

Format