allow subdir configuration for bib notes

[bdarcus]

This is a followup to this back-and-forth.

This makes me wonder: could/should not org-roam-bibtex use the bibtex-completion-notes-path variable if it's set?

I started to draft a new config file for my doom installation based on some in-progress work (namely a new biblio module), and I realized that, at least in how I was doing this, I have some unnecessary duplication (the same org-roam subdirectory set in three different places).

[bdarcus]

And/or, define a new org-roam-bibtex-subdir variable, that is a token that represents a subdir of the OR directory.

I know you mentioned setting this in the template @Zaeph, but I don't think that's as clear and transparent to a user.

[zaeph]

Since we're working with the assumption that your bibliographical notes are somewhere inside your org-roam-directory, I don't see why we'd need to also cover bibtex-completion-notes-path, unless I'm missing something?

[bdarcus]

Oh, because one can simply delete the bibtex-completion-notes-path setq?

As in, ideally one could simply let org-roam-bibtex take care of all this without any explicit notes-directory configuration?

What about the org-roam-bibtex-subdir idea?

[zaeph]

What about the org-roam-bibtex-subdir idea?

I'm still waiting on the merge for jethrokuan/org-roam#427. That way, we'll be able to set the directory via :file-name directly in the template.

[myshevchuk]

could/should not org-roam-bibtex use the bibtex-completion-notes-path variable if it's set?

@bdarcus there can hardly be any use of that variable. Well, in principle, one can check if the variable is non-nil and then preformat templates to include the subdirectory cleverly retrieved from bibtex-completion-notes-path. But there would really be very little gain in that, to my opinion, when one can simply put the desired subdirectory into templates.

So to make the architectural things more clear: org-roam-find-file and org-roam-find-ref are responsible for retrieving notes while org-roam-capture-templates and org-roam-bibtex-template are responsible for putting the files into subdirectories if any. Here, none of the bibtex-completion-notes functionality is involved. We are shadowing any of it by design. bibtex-completion does its primary job: retrieving bibliographic information and org-roam does its job: dealing with the notes.

There is actually not that much sense in making any subdirectories at all. Keep it easy, what will you anyway do when the number of notes grows to hundreds and thousands?

[myshevchuk]

Oh, because one can simply delete the bibtex-completion-notes-path setq?

Yes. Only org-roam-directory is relevant.

As in, ideally one could simply let org-roam-bibtex take care of all this without any explicit notes-directory configuration?

It is so by design.

[bdarcus]

There is actually not that much sense in making any subdirectories at all. Keep it easy, what will you anyway do when the number of notes grows to hundreds and thousands?

I'm still open to considering that.

OTOH, these are conceptually fundamentally different kinds of notes than the standard org-roam notes, and this is behind my impulse to separate them.

I suspect I will not be alone.

[zaeph]

What about the org-roam-bibtex-subdir idea?

I'm still waiting on the merge for jethrokuan/org-roam#427. That way, we'll be able to set the directory via :file-name directly in the template.

To come back on that, I misremembered the PR. You can already create sub-directories by having :file-name "sub/file" in your template. Do note that there is a bug upstream when the sub-directory does not exist. PR underway at jethrokuan/org-roam#476.

[myshevchuk]

@bdarcus Well, I do separate them as well. And give them different types of filenames. Just for pure aesthetics and for who knows what future compatibility with whatever is going to come after org-roam and org-roam-bibtex and bibtex-completion.

[bdarcus]

Actually, to expand a bit on my point, I'd go ever further.

@Zaeph was previously noting he was ensuring no duplicate bib notes.

That's good from a UX perspective, because it reinforces the basic conceptual model, which I argue will not be immediately intuitive; the idea that we distinguish between lower-level bib notes, and notes that link (through citations) to the bib notes.

Putting these notes in a separate subdir also reinforces that basic, clean, conceptual distinction.

Finally, I think jethro was considering, or maybe there was a FR over there (I don't remember) having some kind of visual distinction for bib notes.

So I would actually say a subdir should be default, with ability to change its name in a setq.

All of the above would consistently emphasize the basic conceptual model, which is rooted in the fairly well-developed zettlekasten model.

[zaeph]

That's good from a UX perspective, because it reinforces the basic conceptual model, which I argue will not be immediately intuitive; the idea that we distinguish between lower-level bib notes, and notes that link (through citations) to the bib notes.

Putting these notes in a separate subdir also reinforces that basic, clean, conceptual distinction.

I agree. After, I’m not going to go out of my way beyond setting sane default to enforce the distinction. People can do whatever they want with their templates.

Finally, I think jethro was considering, or maybe there was a FR over there (I don't remember) having some kind of visual distinction for bib notes.

That’s jethrokuan/org-roam#427.

So I would actual say a subdir should be default, with ability to change its name in a setq.

I’d also like to force a sub-directory by default to enforce good separation. Sure, we’re using a different slug than vanilla org-roam, but that’s not enough in my eyes.

[myshevchuk]

Well, the ideology of emacs and org-mode and many other packages is not to enforce but to give the user a choice, an option.

What if I want different subdirectories for books, for patents, for articles and for book chapters? What if I want to sort them by author or journal? What if I actually use org-roam-bibtex only for bibliography notes?

I'd rather not decide what is better for the user. That's why there are org-...-templates and a machinery that ensures the templates will be processed in a flexible and consistent manner. And there are convenience functions to help the user retrieve necessary information quickly and reliably. And so on.

Not that I'm against a functionality that would independently of templates allow to sort out the bibliography notes into a separate directory, but in my opinion it should be purely optional, best of all off by default, and in no case should the internal logic depend on whether such a directory is present or not.

[zaeph]

Well, the ideology of emacs and org-mode and many other packages is not to enforce but to give the user a choice, an option.

You're right, and I've phrased my previous message poorly. I used the word 'force' and 'enforce' when I merely meant to suggest a sane default.

I think it's a good idea to place notes in a ref/ sub-directory by default if we make it obvious in README.md, but we could just as well place them in the root of org-roam-directory and explain that you can customise it. Either way is fine with me.

[myshevchuk]

Agree, that is a justified approach, to set some opinionated defaults and clearly outline the user options.

[bdarcus]

I think this thread evolved to a reasonable compromise, but just want to comment on this for clarification.

What if I want different subdirectories for books, for patents, for articles and for book chapters?

I thought about this as well yesterday, but I think it's a non-issue. What we're talking about here is a way to reinforce a fundamental conceptual distinction in the basic model. Your example here is a completely different issue, which is separating by bib note category.

What if I want to sort them by author or journal? What if I actually use org-roam-bibtex only for bibliography notes?

Obviously you'd want to allow that. But it wouldn't make any sense at all, as the whole point of the zettelkasten system is NOT to do that, but to integrate those bib notes in a larger systems of linked notes.

I'm really just saying, to go with the flow of this conversation, that "reasonable defaults" that clearly distinguish the basic conceptual model, and so help users understand how it's intended to be used are a good idea.

I would be fine with an optional variable to do this, and a note in the documentation that says why you might want to do this and how. Maybe that could include a sentence or two about the zettlekasten conceptual modeling here?

I still get confused about this modeling, BTW, as witnessed by this discussion from yesterday.

https://github.com/jethrokuan/org-roam/issues/474

[bdarcus]

I think this is related:

https://github.com/org-roam/org-roam/pull/604

[bdarcus]

With the new tags functionality, I guess this is already supported.

But maybe we could use a variable for the bib tag name?

[myshevchuk]

@bdarcus could you please elaborate?

[bdarcus]

I haven't fully though this through, as this may not be necessary, but ...

I wanted a way to type notes as bib notes, and put them in a subdir, to clearly distinguish them from other notes.

The new tags support in org-roam adds a general tagging feature which does this.

https://org-roam.github.io/org-roam/manual/Tags.html#Tags

Here tags can either be explicitly added to the #+ROAM_TAGS property, or (optionally) by placing them in a named subdir; so that org-roam/biblio/test.org would be tagged in the system with "biblio."

So as I was meaning here, we could have a variable for that tag, and if the user had the "all-directories" tag option set, then ORB notes would go in that subdir.

Otherwise, it would add the tag to the header.

[myshevchuk]

@bdarcus If it's just about putting notes into a separate directory, one can simply adjust the orb-templates accordingly (or org-roam-capture-templates)

(setq orb-templates '(("r" "ref" plain #'org-roam-capture--get-point
                         "%?" 
                         :file-name "SUBDIRECTORY/${citekey}"
                         :head "#+TITLE: ${title}\n#+ROAM_KEY: ${ref}\n"
                         :unnarrowed t))

[jmburgos]

I am glad I found this discussion because I could not figure out why org-roam-bibtex was ignoring the path in bibtex-completion-notes-path and placing literature notes in the org-roam base directory, and not in my literature sub-folder.

I see now that I can specify the directory of the literature notes in the template. Perhaps we should mention in the documentation that this is the intended behaviour? In particular for people that come to org-roam and org-roam-bibtex already using org-ref and/or helm-bibtex, and that have the path to their literature notes already set using bitbex-completion-notes-path and/or org-ref-notes-directory.

[zaeph]

Yes, I agree. I thought we were tracking this in an issue, but that might have been on Discourse instead. At any rate, there's going to be an effort towards documentation in the 2nd half of June to make sure that we're documenting everything that needs to be.