Discovering / repairing broken references after moving files

[chrischambers]

Brief Abstract

When moving a file into a different subdirectory under org-roam-directory (via, say, a file manager like OSX Finder or git mv) or when renaming a file using the terminal, perhaps, it would be excellent if org-roam could detect the changes and prompt me to update the broken references.

Proposed Implementation

This would require a slightly more extensive change than https://github.com/jethrokuan/org-roam/pull/124 - you'd need some kind of heuristic. I see you're already storing the file hash under roam-db:files - perhaps a job which traverses org-roam-directory for matching hashes but different filenames would suffice?

Please check the following:

• [x] No similar feature requests

[chrischambers]

Awesome project, by the way. :)

[asymmetric]

I personally don't think org-roam should become a kitchen sink of all possible useful features.

Couldn't this functionality be implemented more generically for any org-mode file?

[chrischambers]

Here's a usecase to explain why this isn't just a "kitchen sink" feature:

I start off creating some programming notes - I make a note on the solid principles, another on tdd, another on bdd, one as a reference for elisp, another as a reference for scheme, and so on. In the meantime I'm creating backlinks between them - all is good.

But now I have enough programming notes to warrant creating a new subdirectory so they aren't mixed up with my other notes - so I move them via the terminal or vifm or some such. Now all the backlinks are broken. I have to do a lot of careful grepping to find out which file links are broken, and then maybe a brittle sed -i command to update them. Assuming I remember to do that.

Time goes on and I want to create new subdirectories - one on processes (where solid_principles.org might go), another on testing (where the tdd/bdd notes go), another for languages (scheme, elisp, etc.). If every time I do this all my backlinks get broken, the system won't be sustainable. I think it's a pretty common usecase.

As to whether this could be a generic org-mode feature - I'm not sure about that. It only works presuming there's a finite set of locations/location (e.g. org-roam-directory) under which all the interlinked files sit. Some people choose not to collect all their org notes under org-directory. It also seems easier to implement for org-roam than org in general because of the sqlite db storing file hashes. Perhaps it could start off as an roam feature and then be migrated out into a discrete org module, assuming it's feasible?

[asymmetric]

Thanks for the usecase :)

I’ll only say that to me, this idea of having directories representing categories seems antithetical to the Roam approach of having all notes live in a flat hierarchy, and have the relationships between the notes kind of emerge and still be fluid.

What’s the benefit of dividing things into directories really? Couldn’t you use org-roam tags to group e.g. processes?

[jethrokuan]

I actually echo @asymmetric 's opinion. I don't think it's a good idea to introduce a hierarchy using the filesystem.

If you do want to ensure that file links dont break, I think it's best you use something like Dired to move the files around.

That said, I think there needs to be an org-roam-doctor command. Org-roam should help keep the notes directory clean, that includes detecting broken file links. That will need to wait until the core features are stable.

[space-pope]

Rather than opening a new issue, this one seems like a natural place to bring up a question I've had: I found org-roam via your blog posts on org-mode and zettelkasten, @jethrokuan. They're much appreciated, and I like the tool so far, so thanks! Your last post on zettelkasten, though, mentions a taxonomy of notes, which I also liked—separating papers, talks, and projects just to keep things a bit cleaner. It seems, though, that you've had a change of heart on that approach? Is your personal knowledgebase still being updated from a structure like that, or is everything completely flat now?

[jethrokuan]

Is your personal knowledgebase still being updated from a structure like that, or is everything completely flat now?

It's flat, but the web export still has that hierarchy.

[space-pope]

Gotcha. So in order to maintain that, the file names themselves must end up with projects_, etc prefixes, or is that done with org properties/keywords?

[jethrokuan]

It's a tag for Hugo: #+HUGO_SECTION. Id say that it's fine to have the files in a hierarchy, but it shouldn't be topical, or have remotely any connection to how the notes are connected

[chrischambers]

I actually echo @asymmetric 's opinion. I don't think it's a good idea to introduce a hierarchy using the filesystem.

I understand why you'd say that, but I would (humbly) suggest that org-roam should probably facilitate this. The whole folders vs. tags debate has been going back and forth for years, and the only thing I've found that works for me personally is to use systems which have both as first-class citizens, and which therefore let the user make the decision about where/when it's best to employ either. There are definitely use-cases where discrete bins help organisation and understanding rather than hinder it.

[natejlong]

You all may have thought about this already:

What about leaning on org-id to attach uuids to headlines, and then linking based on that? https://github.com/l3kn/org-zettelkasten does something like this.

Pros:

• file structure doesn't matter, now or later
• easy to split files up while preserving all links to appropriate headlines

Cons:

• All linkable headlines need IDs

What I like about this approach is that it keeps friction low when writing notes. One doesn't need to worry about future organization, and if/when it makes sense to re-organize notes later on the system continues to just work.

[bdarcus]

@natejlong - two things, just thinking out loud:

1. how does that address this issue? what if you don't have any headings in your note?
2. wouldn't that conflict with the atomicity goal of the zettelkasten system, and make the UX more complicated? is that additional complexity really worth it?

[natejlong]

Thanks for the response, @bdarcus

how does that address this issue? what if you don't have any headings in your note?

It doesn't directly address this, and you're right about notes lacking headings. I've been looking around at the different zettelkasten implementations for emacs and thought the strategy was worth bringing up as an idea that others have used to remove this class of problems.

wouldn't that conflict with the atomicity goal of the zettelkasten system, and make the UX more complicated? is that additional complexity really worth it?

It certainly swaps files for headlines as the core object. I personally don't think that's a negative, and I think it wouldn't be too out of place considering how org-roam already is built around the org-mode environment. It would certainly mean more work than simply leaving things as they are, though.

I don't want to distract from the main issue so I'll leave this off, just wanted to throw the idea into the ring!