search

meedstrom / org-node

A notetaking system like Roam using Emacs Org-mode
GNU General Public License v3.0
165 stars 6 forks source link

readme

+TITLE: org-node

+AUTHOR: Martin Edström

+EMAIL: meedstrom91@gmail.com

+STARTUP: content

+TEXINFO_DIR_CATEGORY: Emacs

+TEXINFO_DIR_TITLE: Org-node: (org-node).

+TEXINFO_DIR_DESC: Link org-id entries into a network.

+EXPORT_FILE_NAME: org-node

+HTML: MELPA MELPA Stable

I like [[https://github.com/org-roam/org-roam][org-roam]] but found it too slow, so I made [[https://github.com/meedstrom/quickroam][quickroam]]. And that idea spun off into this package, a standalone thing. It may also be easier to pick up than org-roam.

** What's a "node"?

My life can be divided into two periods "before org-roam" and "after org-roam". I crossed a kind of gap once I got a good way to link between my notes. It's odd to remember when I just relied on browsing subtrees and filesystem directories -- what a strange way to work!

I used to lose track of things I had written, under some forgotten heading in a forgotten file in a forgotten directory. The org-roam method let me find and build on my own work, instead of [[https://en.wikipedia.org/wiki/Cryptomnesia][recreating it all the time]].

At the core, all the "notetaking packages" ([[https://github.com/rtrppl/orgrr][orgrr]]/[[https://github.com/localauthor/zk][zk]]/[[https://github.com/EFLS/zetteldeft][zetteldeft]]/[[https://github.com/org-roam/org-roam][org-roam]]/[[https://github.com/protesilaos/denote][denote]]/[[https://github.com/kaorahi/howm][howm]]/[[https://github.com/kisaragi-hiu/minaduki][minaduki]]/...) try to help you with this: make it easy to link between notes and explore them.

Right off the bat, that imposes two requirements: a method to search for notes, since you can't link to something you can't search for, and a design-choice about what kinds of things should turn up as search hits. What's a "note"?

Just searching for Org files is too coarse. Just searching for any subtree anywhere brings in too much clutter.

Here's what org-roam invented. It turns out that if you limit the search-hits to just those files and subtrees you've deigned to assign an org-id -- which roughly maps to /everything you've ever thought it was worth linking to/ -- it filters out the noise excellently.

Once a subtree has an ID you can link to, it's a "node" because it has joined the wider graph, the network of linked nodes. I wish the English language had more distinct sounds for the words "node" and "note", but to clarify, I'll say "ID-node" when the distinction matters.

** Features

A comparison of three similar systems, which permit relying on org-id and don't lock you into concept of "one-note-per-file".

| Feature | org-roam | org-node | [[https://github.com/toshism/org-super-links][org-super-links]] | |--------------------------------+----------+--------------------+----------------------| | Backlinks | yes | yes | yes | | Node search and insert | yes | yes | -- (suggests [[https://github.com/alphapapa/org-ql][org-ql]]) | | Node aliases | yes | yes | -- | | Node exclusion | yes | limited | not applicable | | Refile | yes | yes | -- | | Rich backlinks buffer | yes | yes (org-roam's) | -- | | Customize how backlinks shown | yes | yes (org-roam's) | yes | | Reflinks | yes | yes (as backlinks) | -- | | Ref search | yes | yes (as aliases) | not applicable | | Org 9.5 @citations as refs | yes | yes | not applicable | | Support org-ref v3 | yes | limited | not applicable | | Support org-ref v2 | yes | -- | not applicable | | Work thru org-roam-capture | yes | yes | ? | | Work thru org-capture | -- | yes | ? | | Daily-nodes | yes | yes | -- | | Node sequences | -- | yes | -- | | Show backlinks in same window | -- | yes | yes | | Cooperate with org-super-links | -- | yes | not applicable | | Fix link descriptions | -- | yes | -- | | List dead links | -- | yes | -- | | Rename file when title changes | -- | yes | -- | | Warn about duplicate titles | -- | yes | -- | | Principled "related-section" | -- | -- | yes | | Untitled notes | -- | -- | -- | | Support =roam:= links | yes | -- (WONTFIX) | not applicable | | Can have separate note piles | yes | -- (WONTFIX) | not applicable | |--------------------------------+----------+--------------------+----------------------| | Some query-able cache | EmacSQL | hash tables | -- | | Async cache rebuild | -- | yes | not applicable | | Time to cache my 3000 nodes | 2m 48s | 0m 01s | not applicable | | Time to save file w/ 400 nodes | 5--10s | instant | ? | | Time to open minibuffer | 1--3s | instant | not applicable |

Assuming your package manager knows about [[https://melpa.org/#/getting-started][MELPA]], add this initfile snippet:

+begin_src elisp

(use-package org-node :after org :config (org-node-cache-mode))

+end_src

If you use org-roam, you probably want the following module as well. Check its README to make org-node [[https://github.com/meedstrom/org-node-fakeroam][work with org-roam side-by-side]]!

+begin_src elisp

(use-package org-node-fakeroam :defer)

+end_src

*** An update broke things? You can rollback to a version that used to work. See [[https://github.com/meedstrom/org-node#appendix-ii-how-to-rollback][How to rollback]] at the end of this readme.

As of v1.8, part of the code was spun out to a new library ([[https://github.com/meedstrom/el-job/][el-job]]) -- you may have to update your package repositories via commands =package-refresh-contents=, =elpaca-update-menus= or equivalent.

** Quick start

If you're new to these concepts, fear not. The main things for day-to-day operation are two verbs: "find" and "insert".

Pick some short keys and try them out.

+begin_src elisp

(keymap-set global-map "M-s M-f" #'org-node-find) (keymap-set org-mode-map "M-s M-i" #'org-node-insert-link)

+end_src

To browse config options, type =M-x customize-group RET org-node RET=.

Final tip: there's no separate command for creating a new node! Reuse one of the commands above, and type the name of a node that doesn't exist. Try it and see what happens!

** Backlink solution 1: Borrow org-roam's backlink buffer As a Roam user, you can keep using =M-x org-roam-buffer-toggle=.

TIP: If it has been slow, or saving files has been slow, [[https://github.com/meedstrom/org-node-fakeroam][org-node-fakeroam]] gives you ways to speed it up.

TIP: If you have not done so yet, I recommend binding some short key sequences. I spent many months waffling on where to bind them, so here's an example:

+begin_src elisp

;; Either this... (keymap-set org-mode-map "M-s M-r" #'org-roam-buffer-toggle) (keymap-set global-map "M-s M-d" #'org-roam-buffer-display-dedicated)

;; ...or just this for a different behavior (keymap-set global-map "M-s M-r" #'org-node-fakeroam-show-buffer)

+end_src

** Backlink solution 2: Print inside the file I rarely have the screen space to display a backlink buffer. Because it needs my active involvement to keep visible, I go long periods seeing no backlinks. This solution can be a great complement (or even stand alone).

*** Option 2A: Let org-node manage a =:BACKLINKS:= property

For a first-time run, type =M-x org-node-backlink-fix-all-files=. (Don't worry if you change your mind; undo with =M-x org-node-backlink-regret=.)

Then enable the following global mode, which keeps these properties updated.

+begin_src elisp

(org-node-backlink-mode)

+end_src

NOTE 1: To be clear, this never generates new IDs. That's your own business. This only adds/edits :BACKLINKS: properties, and no backlink will appear that correspond to a link if the context for that link has no ID among any outline ancestor.

NOTE 2: By default, the setting =org-node-backlink-aggressive= is nil, so that stale backlinks are not cleaned until you carry out some edits under an affected heading and then save the file, which fixes that heading's :BACKLINKS: property. If you'd like it to be more proactive, set t:

+begin_src elisp

(setq org-node-backlink-aggressive t)

+end_src

NOTE 3: People who prefer to hard-wrap text instead of enabling =visual-line-mode= or similar may not find this way of displaying backlinks very scalable, since Org places properties on a single logical line, that may run off the edge of the screen.

*** Option 2B: Let org-super-links manage a =:BACKLINKS:...:END:= drawer

I /think/ the following should work. Totally untested, let me know!

+begin_src elisp

(add-hook 'org-node-insert-link-hook #'org-node-convert-link-to-super)

+end_src

Bad news: this is currently directed towards people who used [[https://github.com/toshism/org-super-links][org-super-links]] from the beginning, or people who are just now starting to assign IDs, as there is not yet a command to add new BACKLINKS drawers in bulk to preexisting nodes. ([[https://github.com/toshism/org-super-links/issues/93][org-super-links#93]])

Ever run into "ID not found" situations? Org-node gives you an extra way to feed data to org-id, as [[http://edstrom.dev/wjwrl/taking-ownership-of-org-id][I find clumsy the built-in options]].

Example setting:

+begin_src elisp

(setq org-node-extra-id-dirs '("~/org/" "~/Syncthing/" "/mnt/stuff/"))

+end_src

Do a =M-x org-node-reset= and see if it can find your notes now.

*** Undoing a Roam hack

If you have org-roam loaded, but no longer update the DB, opening a link can sometimes send you to an outdated file path due to [[https://github.com/org-roam/org-roam/blob/2a630476b3d49d7106f582e7f62b515c62430714/org-roam-id.el#L91][a line in org-roam-id.el]] that causes org-id to /preferentially/ look up the org-roam DB instead of org-id's own table!

Either revert that with the following snippet, or if [[https://github.com/meedstrom/org-node-fakeroam][Fakeroam]] can cover your needs, simply delete the Roam DB (at "~/.emacs.d/org-roam.db").

+begin_src elisp

;; Undo a Roam override (with-eval-after-load 'org-roam-id ;; Restore default for Org 9.1 through 9.7+ (org-link-set-parameters "id" :follow #'org-id-open :store #'org-id-store-link-maybe))

+end_src

** Exclude uninteresting nodes

One user had over a thousand project-nodes, but only just began to do a knowledge base, and wished to avoid seeing the project nodes.

This could work by, for example, excluding a "project" tag and excluding any note that has a TODO state:

+begin_src elisp

(setq org-node-filter-fn (lambda (node) (not (or (org-node-get-todo node) (member "project" (org-node-get-tags-with-inheritance node)) (assoc "ROAM_EXCLUDE" (org-node-get-properties node))))))

+end_src

Or you could go with a whitelist approach, to see only nodes from a certain directory we'll call "my-wiki":

+begin_src elisp

(setq org-node-filter-fn (lambda (node) (and (string-search "/my-wiki/" (org-node-get-file-path node)) (not (assoc "ROAM_EXCLUDE" (org-node-get-properties node))))))

+end_src

(NB: if you don't know what =ROAM_EXCLUDE= is, feel free to omit that clause)

*** Limitation: =ROAM_EXCLUDE=

Let's say you have a big archive file, fulla IDs, and you want all the nodes within out of sight.

Putting a =:ROAM_EXCLUDE: t= at the top won't do it, because unlike in org-roam, child ID nodes of an excluded node are not excluded! The =org-node-filter-fn= applies its ruleset to each node in isolation.

However, nodes in isolation do still have inherited tags. So you can exploit that, or the outline path or file name.

Really, filename! Even though a big selling point of IDs is that you avoid depending on filenames, it's pragmatic to let up on principles sometimes :-) It works well for me to filter out any file or directory that happens to contain "archive" in the name:

+begin_src elisp

(setq org-node-filter-fn (lambda (node) (not (string-search "archive" (org-node-get-file-path node)))))

+end_src

Or put something like =#+filetags: :hide_node:= at the top of each file, and set:

+begin_src elisp

(setq org-node-filter-fn (lambda (node) (not (member "hide_node" (org-node-get-tags-with-inheritance node))))))

+end_src

** Org-capture

You may have heard that org-roam has a set of meta-capture templates: the =org-roam-capture-templates=.

People who understand the magic of capture templates, they may take this in stride. Me, I never felt confident using a second-order abstraction over an already leaky abstraction I didn't fully understand.

Can we just use vanilla org-capture? That'd be less scary. The answer is yes!

The secret sauce is =(function org-node-capture-target)=:

+begin_src elisp

(setq org-capture-templates '(("i" "Capture into ID node" plain (function org-node-capture-target) nil :empty-lines-after 1)

    ("j" "Jump to ID node"
     plain (function org-node-capture-target) nil
     :jump-to-captured t
     :immediate-finish t)

    ;; Sometimes handy after `org-node-insert-link', to
    ;; make a stub you plan to fill in later, without
    ;; leaving the current buffer for now
    ("s" "Make quick stub ID node"
     plain (function org-node-capture-target) nil
     :immediate-finish t)))

+end_src

With that done, you can optionally configure the everyday commands =org-node-find= & =org-node-insert-link= to outsource to org-capture when they try to create new nodes:

+begin_src elisp

(setq org-node-creation-fn #'org-capture)

+end_src

That last optional functionality may be confusing if I describe it -- better you try and see if you like.

** Completion-at-point To complete words at point into known node titles:

+begin_src elisp

(org-node-complete-at-point-mode) (setq org-roam-completion-everywhere nil) ;; Prevent Roam's variant

+end_src

** FAQ: Any analogue to =org-roam-node-display-template=?

Want to customize how the nodes look in the minibuffer?

Configure =org-node-affixation-fn=:

: M-x customize-variable RET org-node-affixation-fn RET

A related variable is =org-node-alter-candidates=. Read its docstring.

** Grep

If you have Ripgrep installed on the computer, and [[https://github.com/minad/consult][Consult]] installed on Emacs, you can use this command to grep across all your Org files at any time.

+begin_src elisp

(keymap-set global-map "M-s M-g" #'org-node-grep)

+end_src

This can be a power tool for mass edits. Say you want to rename some Org tag =:math:= to =:Math:= absolutely everywhere. Then you could follow a procedure such as:

  1. Use =org-node-grep= and type =:math:=
  2. Use =embark-export= (from package [[https://github.com/oantolin/embark][Embark]])
  3. Use =wgrep-change-to-wgrep-mode= (from package [[https://github.com/mhayashi1120/Emacs-wgrep][wgrep]])
  4. Do a query-replace (~M-%~) to replace all =:math:= with =:Math:=
  5. Type ~C-c C-c~ to apply the changes

** Let org-open-at-point detect refs

Say there's a link to a web URL, and you've forgotten you also have a node listing that exact URL in its =ROAM_REFS= property.

Wouldn't it be nice if, clicking on that link, you automatically visit that node first instead of being sent to the web? Here you go:

+begin_src elisp

(add-hook 'org-open-at-point-functions

'org-node-try-visit-ref-node)

+end_src

** Limitation: TRAMP Working with files over TRAMP is unsupported for now. Org-node tries to be very fast, often nulling =file-name-handler-alist=, which TRAMP needs.

The best way to change this is to [[https://github.com/meedstrom/org-node/issues][file an issue]] to show you care :-)

** Limitation: Encryption Encrypted nodes probably won't be found. As with TRAMP, file an issue.

** Limitation: Unique titles If two ID-nodes exist with the same title, one of them disappears from minibuffer completions.

That's just the nature of completion. Much can be said for embracing the uniqueness constraint, and org-node will print messages about collisions.

Anyway... there's a workaround. Assuming you leave =org-node-affixation-fn= at its default setting, adding this to initfiles tends to do the trick:

+begin_src elisp

(setq org-node-alter-candidates t)

+end_src

This lets you match against the node outline path and not only the title, which resolves most conflicts given that the most likely source of conflict is subheadings in disparate files, that happen to be named the same. [[https://fosstodon.org/@nickanderson/112249581810196258][Some people]] make this trick part of their workflow.

NB: for users of =org-node-complete-at-point-mode=, this workaround won't help those completions. With some luck you'll rarely insert the wrong link, but it's worth being aware. ([[https://github.com/meedstrom/org-node/issues/62][#62]])

** Limitation: Org-ref

Org-node supports the Org 9.5 @citations, but not fully the aftermarket [[https://github.com/jkitchin/org-ref][org-ref]] &citations that emulate LaTeX look-and-feel, because it would double the time taken by =M-x org-node-reset=.

What works is bracketed Org-ref v3 citations that start with "cite", e.g. =[[citep:...]]=, =[[citealt:...]]=, =[[citeauthor:...]]=, since org-node-parser.el is able to pick them up for free.

What doesn't work is e.g. =[[bibentry:...]]= since it doesn't start with "cite", nor plain =citep:...= since it is not wrapped in brackets.

If you need more of Org-ref, you have at least two options:

** Toolbox

Basic commands:

Rarer commands:

** Experimental: Node sequences Do you already know about "daily-notes"? Then get started with a keybinding such as:

+begin_src elisp

(keymap-set global-map "M-s s" #'org-node-seq-dispatch)

+end_src

and configure =org-node-seq-defs=. See [[https://github.com/meedstrom/org-node/wiki/Configuring-series][wiki]].

*** What are "node seqs"? It's easiest to explain node sequences if we use "daily-notes" (or "dailies") as an example.

Roam's idea of a "daily-note" is the same as an [[https://github.com/bastibe/org-journal][org-journal]] entry: a file/entry where the title is just today's date.

You don't need software for that basic idea, only to make it extra convenient to navigate them and jump back and forth in the series.

Thus, fundamentally, any "journal" or "dailies" software are just operating on a sorted series to navigate through. A node sequence. You could have sequences for, let's say, historical events, Star Trek episodes, your school curriculum...

Currently, defining a new node seq requires writing 5 lambdas, but once you get the hang of it, you can often reuse those lambdas.

*** Future A future version will likely bring convenient wrappers that let you define a node seq in 1-2 lines.

It's also possible we just redesign this completely. Input welcome. How would you like to define a node sequence? Where should the information be stored?

API cheatsheet between org-roam and org-node.

| Action | org-roam | org-node | |-----------------------------------------+----------------------------------+-------------------------------------------------------------------| | Get ID near point | =(org-roam-id-at-point)= | =(org-entry-get-with-inheritance "ID")= | | Get node at point | =(org-roam-node-at-point)= | =(org-node-at-point)= | | Get list of files | =(org-roam-list-files)= | =(org-node-list-files)= | | Prompt user to pick a node | =(org-roam-node-read)= | =(org-node-read)= | | Get backlink objects | =(org-roam-backlinks-get NODE)= | =(org-node-get-id-links-to NODE)= | | Get reflink objects | =(org-roam-reflinks-get NODE)= | =(org-node-get-reflinks-to NODE)= | | Get title | =(org-roam-node-title NODE)= | =(org-node-get-title NODE)= | | Get title of file where NODE is | =(org-roam-node-file-title NODE)= | =(org-node-get-file-title NODE)= | | Get title /or/ name of file where NODE is | | =(org-node-get-file-title-or-basename NODE)= | | Get name of file where NODE is | =(org-roam-node-file NODE)= | =(org-node-get-file-path NODE)= | | Get ID | =(org-roam-node-id NODE)= | =(org-node-get-id NODE)= | | Get tags | =(org-roam-node-tags NODE)= | =(org-node-get-tags-with-inheritance NODE)= | | Get local tags | | =(org-node-get-tags-local NODE)= | | Get outline level | =(org-roam-node-level NODE)= | =(org-node-get-level NODE)= | | Get whether this is a subtree | =(=< 0 (org-roam-node-level NODE))= | =(org-node-get-is-subtree NODE)= | | Get char position | =(org-roam-node-point NODE)= | =(org-node-get-pos NODE)= | | Get properties | =(org-roam-node-properties NODE)= | =(org-node-get-properties NODE)=, only includes explicit properties | | Get subtree TODO state | =(org-roam-node-todo NODE)= | =(org-node-get-todo NODE)= | | Get subtree SCHEDULED | =(org-roam-node-scheduled NODE)= | =(org-node-get-scheduled NODE)= | | Get subtree DEADLINE | =(org-roam-node-deadline NODE)= | =(org-node-get-deadline NODE)= | | Get subtree priority | =(org-roam-node-priority NODE)= | =(org-node-get-priority NODE)= | | Get outline-path | =(org-roam-node-olp NODE)= | =(org-node-get-olp NODE)= | | Get =ROAM_REFS= | =(org-roam-node-refs NODE)= | =(org-node-get-refs NODE)= | | Get =ROAM_ALIASES= | =(org-roam-node-aliases NODE)= | =(org-node-get-aliases NODE)= | | Get =ROAM_EXCLUDE= | | =(assoc "ROAM_EXCLUDE" (org-node-get-properties NODE))= | | Ensure fresh data | =(org-roam-db-sync)= | =(org-node-cache-ensure t t)= |

** Appendix II: How to rollback

Instructions to downgrade to [[https://github.com/meedstrom/org-node/tags][an older version]], let's say 1.6.2.

With [[https://github.com/quelpa/quelpa][Quelpa]]:

+begin_src elisp

(use-package org-node :quelpa (org-node :fetcher github :repo "meedstrom/org-node" :branch "v1.6"))

+end_src

With [[https://github.com/slotThe/vc-use-package][vc-use-package]] on Emacs 29:

+begin_src elisp

(use-package org-node :vc (:fetcher github :repo "meedstrom/org-node" :branch "v1.6"))

+end_src

With built-in =:vc= on Emacs 30+:

+begin_src elisp

(use-package org-node :vc (:url "https://github.com/meedstrom/org-node" :branch "v1.6"))

+end_src

With [[https://github.com/progfolio/elpaca][Elpaca]] as follows. Note that recipe changes only take effect after you do =M-x elpaca-delete= and it re-clones -- the idea is that Elpaca users will prefer to do it manually inside the cloned repo.

+begin_src elisp

(use-package org-node :ensure (:fetcher github :repo "meedstrom/org-node" :branch "v1.6"))

+end_src

...Elpaca can also target an exact version tag. Package manager of the future, it is:

+begin_src elisp

(use-package org-node :ensure (:fetcher github :repo "meedstrom/org-node" :tag "1.6.2"))

+end_src

With [[https://github.com/radian-software/straight.el][Straight]]:

+begin_src elisp

(use-package org-node :straight (org-node :type git :host github :repo "meedstrom/org-node" :branch "v1.6"))

+end_src

*** Match fakeroam version The extension [[https://github.com/meedstrom/org-node-fakeroam][org-node-fakeroam]] has been tightly coupled with contemporary versions of org-node, but hopefully I've made it warn about any version mismatch at load-time.

Just in case, the release history:

*** What version is installed? This info depends on your package manager. I do not embed version in the source code.