(feature-request): edit transclusion without leaving the transcluding buffer

[natask]

I want to start of by saying that I really like the work you have done here.

Now on to the feature request, It would be more convenient if edits could be conducted from within transcluding buffer. It would make it easier to see and possibly switch other content within transcluding buffer. Not a high priority because it is possible to open the edit buffer and the transcluding buffer side by side but would nevertheless have a positive impact on workflow.

To guard against multiple transclusions editing at the same time, either make it so that only one can be edited at the same time or each change needs to reflect in each visible transclusion before another character can be inserted.

[nobiot]

Hi @natask , Thank for your kind words!

It's good to be able to confirm someone values the ability to edit and sync from the transcluding buffer.

I have a mixed feeling about it at the moment, to be honest. This was the default way in v0.0.1, and I tried a real-time sync with v0.0.2.

I pivoted with v0.0.4 away from this sync paradigm in favour of the current way via the indirect buffer.

As a user/writer, I did not find it essential to have the ability to edit the source in a transcluding buffer (though I agree that it feels more convenient for the writer).

This pivot has opened up different opportunities: e.g. the ability to filter out elements that you specify (v0.0.5 feature), and adjust the headline levels (partially available with v0.0.4; I'm looking to extend it in the next update).

In other words, because I don't have to sync FROM the transcluding buffer TO the source, I have more freedom to manipulate a source content without changing it, and render it in the transcluding buffer in ways suitable for assembly. To me, this is essential in the "assemble to edit" paradigm I am experimenting with Org-transclusion; whereas the real-time sync was rather a matter of convenience.

These are my current justifications for my pivot; it does not necessarily mean I won't review the "sync paradigm" in the future. In fact, each new update affords me with a new vantage point where I can see things that I could not see in the previous step -- it sounds like an analogy, but it is quite concrete and real to me, who is in the midst of the experience and development.

Sorry, I might have digressed. Let us consider this feature request in the future. Thank you again for your feedback and raising this feature request.

nobiot

[vv111y]

Hi @nobiot I am interested in this as well. Currently I was looking at using
after-change-functions
for syncing. For my workflow some kind of syncing likely needs to be done.

Indirect buffers, I wish I could use because they are almost exactly the right thing; especially since it is implemented in C. The problem though is shaping the org tree: some headings need to be added/removed, and repositioned. If you can figure out a way to do that I would be deeply grateful. Otherwise I will likely continue with my own syncing implementation.

[nobiot]

@vv111y, thank you. It's an interesting idea.

I would like to explore this problem a bit more. Hope you could help me understand your idea better. Can you elaborate on what you can do with "syncing" but you cannot do with indirect buffers?

I do not fully understand what you mean by "shaping the org tree". The way I understand it is that you want to include or exclude certain elements when you transclude an org file. It's currently implemented via the filter mechanism. And I could do this because I use indirect buffers (the sequence is from the source to the transclusion copies). If I were to "sync" from a transclusion copy to its respective source, I would struggle.

So... If I understand your idea of "sync", "shaping the org tree", and indirect buffers correctly, I have totally the opposite problem. I can shape the org tree with indrect buffers.... I feel that my understanding is incorrect, and therefore my conclusion is incorrect.

[vv111y]

'shaping' wasn't the best word.
I will include and exclude specific subtrees (headers) from a group of org files, not just one. This is done by actual copy&paste into work buffer(s).

Ie. one subtree is the actual note that will be worked on. The others are context material (backlinks, forlinks, etc). These copies and the buffers are understood to be ephemeral; a presentation of knowledge base content with respect to the note being worked on.

The copies are editable which needs to be propagated back to the original sources. The crudest method, which I will use for now, is to recopy whole subtrees and replace the original subtree with the edited version. Using after-change-functions is a refinement.

If I understand the limits of indirect buffers right, the limitations for my use case are:

1. cannot gather content from multiple org files
2. cannot have a hierarchical tree of notes (I refer to them as nodes). I'm breaking with zettlekasten here.
3. cannot make other modifications to the copied presentation that are not meant to be modifications of the source. I can elaborate on that. (specifically edit outside the heading/subtree bounds)

I think there was another reason, can't remember it right now.

Can the filter hide or somehow remove parts of the tree?

[nobiot]

Thank you @vv111y.

I think I have struggled with the same problems, and my answer so far is the way Org-transclusion works with indirect buffers.

Org-transclusion works with headlines (as well as whole files) so you can selectively copy a subtree at any level. If you had this structure below in your source org file, you could choose to copy the whole of H1 subtree, or choose to copy only H1.1.1; in fact any combination, including subtrees overlapping one another (I don't know if that makes sense for the work you do).

You can also have H1.2 separately from H1.1 as a reference, for example, in the same transcluding buffer. It's up to you whether or not you edit the context-only subtree.

* H1
** H1.1
*** H1.1.1
** H1.2 -- Context only
* H2
** H2.1
** H2.2

I specifically chose to use indirect buffers to avoid "recopy whole subtrees and replace the original subtree", because I did not want to risk my code inadvertently deleting my important original notes behind my back due to a bug -- with indirect buffers, you directly edit the source manually; my code won't change it.

I use indirect buffers for editing the source (original). You transclude multiple source subtrees at various levels to assemble a cohesive whole in a separate buffer as read-only copies -- so the limitations you cite there are not really relevant to the way Org-transclusion currently works.

And... with the filters, "Can the filter hide or somehow remove parts of the tree?" This is what the filter function does. Could I invite to you to view this video that I have done to show that?

[vv111y]

So you could return say H1.1.1 and H2.2 only? I agree, doing this kind of syncing makes me nervous. I didn't see heading filtering in the video well done presentation nonetheless.

I'll try this out, see how it goes. How do you specify a list of specific headings that you want returned?

[nobiot]

Yes. It's via the normal Org Mode's file link. To take your example, then... It would be something like this:

# I am in a transcluding org file

Now assuming H1.1.1 and H2.2 are the heading texts (not just the structure)

I want to include H1.1.1 below.
[[file:path/to/file.org::*H1.1.1]]

Some text here.
I want to include H2.2 below
[[file:path/to/file.org::*H2.2]]

You can use ID or CUSTOM-ID to specify the headline/subtree you want to transclude. The demo you saw showed the latter briefly.

I will consider a recap in the beginning of the demo in the next one (hopefully this weekend, or the next). These are in the previous show & tell video. It's a bit long, and not based on the latest version, but you can see how links are supposed to work.

[vv111y]

Ah I see, I misunderstood. The indirect buffer is only for editing.
You are using your own modded org-paste-subtree for content.
This gives me an idea.
Perhaps we should jump to another thread, sorry for hijacking @natask.

[nobiot]

I have also put back a live-sync edit feature as an experiment. You can find more about the new approach and features in the new README.

I would appreciate it you could try the new version and give feedback. Thank you.