Org-mode support

[randomwangran]

Currently, the rendered notes are plain text.

It would be nice to render these notes using Org-mode.

This will be extremely helpful if the notes contain links.

I know Org-transclusion also uses overlay to render some notes. In note overlays, links can be followed.

For example, in the following example, blog is an Org-mode link.

After the notes being loaded, they are rendered as plain text as shown in the following example.

[cage2]

Hi @randomwangran !

I am afraid rendering org mode for notes would be too difficult to implement given my skills. I need some times to understand if and how this could be accomplished.

I would be happy if some users could point me to some resources about how to render org-mode in a single text block.

Bye! C.

[randomwangran]

Not a problem, I will keep this issue open until we can find a solution.

I know the author of Org-transclusion is an expert in overlay rendering. I will try to ask him for helps.

---

Hi @nobiot:

Sorry for bothering you, I know that you are currently working with your book. But we really need your suggestions to work on this issue.

The reason why I bothering with you is that we want to render a standard Org-mode link in an overlay.

I see Org-transclusion is perfectly good at this and I find that org-transclusion--create-at-point extensively use this technology as well.

(defun org-transclusion--create-at-point (tc-params)

  ...

  (overlay-put ov-tc 'tc-type tc-type)
  (overlay-put ov-tc 'tc-raw-link tc-raw-link)
  (overlay-put ov-tc 'tc-beg-mkr tc-beg-mkr)
  (overlay-put ov-tc 'tc-end-mkr tc-end-mkr)

  ...)

@Cage2 and me is trying to implement a standard Org-mode link in an overlay.

I was wondering if you could give us some suggestions to work on this issue.

Thanks for your time.

[nobiot]

hmmm. first of all this is a great package. thank you for bringing it out to the world.

if it is authored by the the creator of org-journal, my humble skill could not be compared.

what i am doing with org-transclusion is very simple. it's essentially copy and paste (via inserting a specific text region). rendering the copied text as org mode is achieved through the buffer being already in org mode.

if you permit me risking a grave misunderstanding on my part, could I ask this question: can you turn on org-mode when you show the "notes", instead of a plain text mode (is it the "fundamental mode"?)?

[randomwangran]

@nobiot

For your question, I've done the following:

1. open annotate.el
2. load org-mode
3. turn-on minor mode annotate-mode instead of emacs-lisp-mode
4. step-3 will load all annotations

But in this case, the annotations are still rendered as fundamental mode, not Org-mode.

I've done the similar things to an Org-mode file contains Org-transclusion links.

After turn-on fundamental mode, I see the link will become a raw string instead of a click-able link.

This is very unique for Org-transclusion being turn on.

Your awesome Org-transclusion package is a testimony for your elisp skill. I use it everyday. There's no doubt.

Thanks for your suggestions on how Org-transclusion renders the content. I will see what I can do.

[nobiot]

@randomwangran , Thank you. Sorry that I don't fully understand all the details, but it sounds to me that "something" explicitly turns on fundamental mode.

1. Can you turn on Org mode instead?
2. Even if Fundamental is on (and the link may not be clickable), Org mode and Org-transclusion might be able to recognize the link syntax within the buffer

As for how Org-transclusion copies and paste a text content:

1. After some checks, it visits the linked buffer (org-transclusion--get-org-content-from-link)
2. Get the content (filter applied) and markers that indicate begin and end of the region (org-transclusion--get-org-buffer-or-element-at-point)
3. Back to the original buffer, insert the content and add overlay (org-transclusion--create-at-point)

It might look complicated, but it's just iteration on one of the earliest versions that does very simply something like this:

1. Visit the linked buffer
2. Copy the whole buffer (point-min and point-max)
3. Insert the whole text and put an overlay on top This is still done for non-Org files with org-transclusion--add-others-default

[randomwangran]

Sorry for not being clear @nobiot. I will try to make it more clear with your questions.

1. Can you turn on Org mode instead?

1. open a file, e.g., annotate.el

   After opening, the major mode will be automatically turned on to emacs-lisp mode

2. load minor mode annotate-mode

   This will load all the annotations associated with the current file. For testing purposes, I've already add an annotation that contains a standard Org-mode link.

3. load major Org-mode to the buffer opening with the file annotate.el

   This will screw up the loaded annotations. This is no a big issue because the data base is still being unchanged. In next two steps, I will reload minor mode annotate-mode, to show the annotations.

4. turn-off minor mode annotate-mode

5. turn-on minor mode annotate-mode

   Now, all the annotations will be loaded and displayed in current buffer.

6. turn on Org-mode to the buffer opening with the file annotate.el

   The expected result would be that the Org-mode link in the annotations will be rendered as the click-able link.

   Unfortunately, the link is still the raw text.

Even if Fundamental is on (and the link may not be clickable), Org mode and Org-transclusion might be able to recognize the link syntax within the buffer

I might miss some here. To make it more clear, I've tested the same buffer with two different modes, i.e., Org-mode and Fundamental-mode.

In the following figure, the buffer is visiting the same file. The first one is in Org-mode. I also use Org-transclusion to include a paragraph that contains an Org-mode link. The red arrow shows that the link is click-able.

Then I turn the major mode to Fundamental-mode, the buffer now looks like the below. As you can see, the region (arrow B) becomes a raw text. It is not a clickable link.

—

Thanks for offering the suggestions on how Org-transclusion copies and paste a text content. I will study it and see what I can do.

[nobiot]

Why do you need to turn on fundamental-mode?

[randomwangran]

Hi @nobiot

Why do you need to turn on fundamental-mode?

This is because I saw your question:

can you turn on org-mode when you show the "notes", instead of a plain text mode (is it the "fundamental mode"?)?

I am trying to see the impact of changing the major mode over the overlay content.

Anyway, I will study your suggestion to see what I can do.

[nobiot]

@randomwangran , I am trying annotate-mode; it looks useful for my learning a new (human) language -- probably not the original intended usage, but good tools can be repurposed, I suppose. I would not need to drag Bastian into this conversation, but thank you for bringing this package out to the world (@cage2 , thank you for actively contributing to the package).

If I understand you correctly, I believe you would like to display a functional Org link in the annotation (you want to be able to click on it to work). I don't think it is really compatible with the design of annotate-mode.

It uses the display property of an overlay (this line). It's the same way the current implementation of roam-block displays the block content. The actual content is not recognized by Emacs -- the text is NOT in the buffer. Therefore, it is not clickable (you cannot move your cursor in the displayed text -- there seems to be a way to do it by way of changing a certain variable, but I don't believe that achieves what you would like).

In implementing org-transclusion I tried this option, too. I decided to do a different way -- insert the text, put an overlay on top, and make the texts read-only, instead of using the display property. The copied text exists in the buffer as a normal text.

This is a fundamental design decision. I could not speak for the Bastian and other contributors of annotate-mode, but I feel that these two different designs are incompatible.

[cage2]

On Sat, Dec 19, 2020 at 04:29:21AM -0800, Noboru Ota wrote:

Hi everyone!

@randomwangran , I am trying annotate-mode; it looks useful for my learning a new (human) language -- probably not the original intended usage, but good tools can be repurposed, I suppose.

I agree, using a tool for scopes the original author was not aware for is a sign that it is useful and in good shape. I hope this continues for annotate in the future. :)

I would not need to drag Bastian into this conversation, but thank you for bringing this package out to the world (@cage2 , thank you for actively contributing to the package).

You are welcome! I am learning a lot from this project and happy to have the opportunity work on it! :)

If I understand you correctly, I believe you would like to display a functional Org link in the annotation (you want to be able to click on it to work). I don't think it is really compatible with the design of annotate-mode.

It uses the display property of an overlay (this line). It's the same way the current implementation of roam-block displays the block content. The actual content is not recognized by Emacs -- the text is NOT in the buffer. Therefore, it is not clickable (you cannot move your cursor in the displayed text -- there seems to be a way to do it by way of changing a certain variable, but I don't believe that achieves what you would like).

In implementing org-transclusion I tried this option, too. I decided to do a different way -- insert the text, put an overlay on top, and make the texts read-only, instead of using the display property. The copied text exists in the buffer as a normal text. This is a fundamental design decision. I could not speak for the Bastian and other contributors of annotate-mode, but I feel that these two different designs are [in]compatible.

I tend to agree too, a fundamental property of annotate is to not modify the buffer and keep separated as much as possible annotations and actual file content. I am not sure could be simple to implement this feature using immutable text in the buffer. Not impossible but this will require a lot of rewriting and -in my opinion, much more complicated, and maybe fragile, code.

By the way, thank you to you all for this interesting discussion! Bye! C.

[randomwangran]

Thank for both of you Nobiot and cage2.

I thought that this was a good feature to bridge the code content to my note database (or Zettelkästen, a slip box to organize personal notes). This is a huge motivation for me. I will probably tend to review more codes and jot down more thoughts. This is definitely not experienced coders will consider to do, but my personal preference.

As @cage2 said:

I am not sure could be simple to implement this feature using immutable text in the buffer.

and @nobiot

I don't think it is really compatible with the design of annotate-mode.

I believe this is really hard to implement given the current code structure. Even if it is implemented, there must have a lot of incompatible features introduced, which I do not want to see.

Nevertheless, I won't easily give it up. The taste of Zettelkästen is irresistible for me.

I will play with myself. Nobiot already gave me a lot of information that I could play with.

In implementing org-transclusion I tried this option, too. I decided to do a different way – insert the text, put an overlay on top, and make the texts read-only, instead of using the display property. The copied text exists in the buffer as a normal text.

I can learn from many other packages like: ov.el, posframe, and roam-block to potentially just make it happen.

Thank again for the discussion to make it clear. I will just close this issue, but I will definitely update it if I have something to share.

[cage2]

Hi!

Nevertheless, I won't easily give it up. The taste of Zettelkästen is irresistible for me.

I think this is the good attitude :+1: , i wish you could reach your goal soon and (why not?) give some more good emacs package too! :smile:

Bye! C.

[randomwangran]

Hi @cage2 :

i wish you could reach your goal soon and (why not?)

@nobiot wrote a new package org-marginalia and I hooked up a posframe for preview purpose.

Though a clickable link in a posframe is not available yet, I find it's okay for me to use it now. The annotations are store in an Org-mode file, so the interaction with Org-roam becomes very easy.

[cage2]

On Thu, Dec 24, 2020 at 02:19:47PM -0800, randomwangran wrote:

Hi @cage2 :

Hi!

i wish you could reach your goal soon and (why not?)

@nobiot wrote a new package org-marginalia and I hooked up a posframe for preview purpose.

So you two wrote code at lighting speed! That is impressive! :)

Though a clickable link in a posframe is not available yet, I find it's okay for me to use it now. The annotations are store in an Org-mode file, so the interaction with Org-roam becomes very easy.

Wow, this looks nice, congratulations! 🎉

Bye! C.

PS: Moreover i learn about posframe, and maybe this could be something annotate could takes advantage of! :)