Feature request: Luhmann-style IDs (for discussion)

[gklimowicz]

I'm experimenting with using zetteldeft and Luhmann-style note IDs in one of my archives. The motivations are

• I like the idea that related notes will tend to sort close to one another.
• It exposes hierarchical structure for (some subset of) the notes.
• When creating or filing a note, I have to more consciously think of where it belongs in the archive.

In his second Zettelkasten, Luhmann had a set of sections and subsections. It seems as though the section numbers (e.g., 1/ at the start of an ID) were well defined, but I'm not sure how he used the subsection numbers. Well, slashes won't work for pathname reasons (I want them in one directory), and section numbers don't work for me, as I'd always be referring to the index to figure out what was what. So, I am using brief identifiers with a dash to define the sections (e.g., "zk-5g Zettelkasten use case: Reading a web page and taking notes" for a use case about taking notes in my Zettelkasten subsection).

My `zetteldeft-id-regex, including the pattern for my previous style of IDs, is

(concat "\\("
        "\\<[12][0-9][0-9][0-9][01][0-9][0-3][0-9]-[0-9][0-9][0-6][0-9]\\>"
        "\\|"
        "\\<[a-z]+-\\([1-9][0-9]*[a-z]*\\)+\\>"
        "\\)")

This discussion is around the best way to handle this sort of ID. Using these kinds of IDs, we

• may want to use the deft index to search for related notes before adding the new note
• may be editing an existing note that this new note will sequence or branch from, or be unrelated to
• may not know exactly where to place the note until after we write it
• probably do know the section we want to put it in (I have about two dozen sections today)
• can't use the existing zetteldeft-generate-id to generate IDs for a new note, but may need to prompt for an ID

Presently, I'm going outside zetteldeft to create the new notes from the shell. It would be nice, though, to be able to do this from within Emacs, deft, and zetteldeft.

[EFLS]

Ha, going by that regex, you've been keeping Zettels for quite some time already :smile:

Interesting thoughts. Some responses:

Not knowing where to put a note from the start is no big deal: use temporary notes and give them a fixed id (and thus a fixed position) later. I think Luhmann called them 'fleeting notes'?

The biggest challenge would be to change how an ID is generated. What would be a good approach? To simply prompt the user to enter one?

[gklimowicz]

Not keeping them so long, but converting old files to Zettels. I'm new here.

Temporary notes: This makes a lot of sense. Even in the current zetteldeft, I can use date-time IDs for the initial draft of the note, then C-c d r the note to the Luhmann-style name with another ID.

The only two things I might want to add to this process are:

1. Ensure that the ID I'm renaming to is unique (because I am not careful);
2. If any other note references the original fleeting note ID, that it gets updated with the new ID (e.g., if created with C-c d N in another note with the fleeting ID).

I'll try to come up with a pull request.

[davidpiano]

If my rambling comment is out of place, please disregard it or delete it. I just started trying Zetteldeft a few minutes ago.

I think my main point is this: Having notes whose links are date-stamps by default might actually be counterproductive in the context of this project. The point as far as I can tell is to structure sets of ideas by their logical associations with each other, and date-stamps can sometimes go against that. I'm not against having date stamps in notes but I think I'm against them as default links, because the links are supposed to be thought links not time links.

To make that very short, there are already many good journal solutions, and IMO this shouldn't be another journal. Imposing a non-time-based hierarchy by default seems essential to me.

After this point, I'm arguing opposite sides against myself because I don't know.

One argument, maybe wrong: From what I've just read above, if the ability to write a note now and categorize it later exists anyway, it's difficult to justify not imposing a hierarchical structure by default. One main point (if not the only point) of the Zettelkasten idea is that a pile of notes - even a pile of notes with tags, even dated notes - is still just a pile.

Another argument, also maybe wrong: The already-existing ability to create links to and from any notes, meaning that structurally any note can be hierarchically several places at once if desired (just by placing its link again), might be already superior to the original paper notes, and might need only to be exposed in an easier-to-manipulate way, such as some kind of rearrangeable structured list. Maybe an org-mode outline that contains just Zettelkasten links? (Even a person who has no interest in org-mode at all can pick up how to manipulate a simple outline structure in two minutes.) It's possible that adding even the ability to form another layer (or type) of hierarchy to this project could just muddy the situation.

Question: Can a person easily force their existing Zetteldeft link names to conform to a rigid hierarchy, if they prefer that?

[davidpiano]

I think I'm suggesting that dates be kept, but permanently demoted from their current prominent position, because navigating a Zettelkasten by date is IMO missing the point. I'd suggest automatically stamping inside the text of each note with the full date and time, but removing it as the ID method.

[EFLS]

Question: Can a person easily force their existing Zetteldeft link names to conform to a rigid hierarchy, if they prefer that?

Not right now. @jeffvalk has a proposal for custom ID generation in #68. From there, Zetteldeft IDs could indeed adhere to specific rules when their generation is customized.

... because navigating a Zettelkasten by date is IMO missing the point.

I agree completely, and the custom IDs are not meant to be navigated directly. Structure, logic or narrative can be generated in different ways than sorting by ID. But these issues are challenges related to digital Zettelkasten. I use automated lists and "structure notes" to apply multiple hierarchies. In the near future, I hope to share my approach in a short video.

[TRSx80]

Interesting discussion. And I should probably read related "custom ID generation" issue that is linked immediately above this comment (and I will after) but so far here are my thoughts on this topic.

I am pretty sure that the reason the ISO-like date stamp has been chosen is because of implementation simplicity. It is very easy to make regex to match it. This opens the door to all manner of advanced automated functionality, mostly around generating and maintaining links.

EFLS link implementation is the "killer feature" of this whole project, IMO. Because you can change the title in the note, and or the title part of file name, without breaking any already existing links. Which is huge once you start really getting going and cross linking a lot of stuff. You don't want to be going back and fixing all those changed links manually (this can be automated, but that also adds complexity).

The ISO-like date stamp has also become almost a sort of "industry standard" in that even other Zettelkasten apps have settled upon that format. That was not on accident, it is because it is a (very) good implementation choice, IMO.

There are some arguments in favor of the date stamp. As mentioned here:

"the Noguchi filing method which acts on the premise that humans tend to remember the date or time of year we did something very well."

But to me that is not even the strongest argument. The strongest argument is the implementation benefits.

Putting this another way, I would suggest not to get hung up on the fact it is a "date" but rather to view it as a very reliable sort UUID generation method (well, down to 1 second resolution, at least, assuming current default format).

As to the implementation of Luhmann-style IDs, I find the concept interesting. I am not trying to discourage it altogether, but rather I think its implementation should live somewhere else than the fundamental ID for the whole system, for the following reasons:

• It is non standard. As already mentioned, many other projects are using this style of "ISO like date as UUID." Which means you have a lot of portability to take your notes and potentially use them in other apps, etc.
• For same reason as above, you are likely to be on your own, implementation and support wise, with such a "nonstandard" format. Maybe some people join you though, who knows... Although I imagine taking such a step would fundamentally alter this project in such a way as to require forking into a new project. Are you up for that amount of work (maybe you are)?
• It makes your fundamental way of locating things (ID) much more complicated and fragile. Compare regex for this style ID to one for vanilla Zetteldeft.

Not saying it shouldn't be done. And there may be some benefits, as you have laid out. But perhaps implemented somehow differently (and I'm not sure what that best way might look like).

Maybe an implementation leveraging Org Property for a secondary "Luhmann-ID"? Then you could use something like org-ql (or even built in Org tools, although org-ql is purportedly faster) to implement whatever you want related to querying the Luhmann ID (checking for dupes, branching, etc...).

Working the other way around, say you go ahead with the whole Luhmann style ID implementation. You could then maintain stable links with something like org-super-links (or even the built-in org-id). Although personally I find those sort of long meaningless UUID quite ugly.

You could also maintain your links using something like projectile-replace to search through your whole Deft folder and replace all occurrences of a particular link with the new link.

Also, all possibilities I mentioned are based in Org/Emacs, because it is a huge ecosystem with a lot of functionality already built for it, which is one of main reasons I also prefer it. But I am sure there are other ways as well, whether in Emacs or whatever other tool(s) you prefer working with.

In any case, keep us updated as to how you are getting on. And good luck!