support for Citar

[jlcolbert]

If using Vertico or Selectrum as completion interfaces, Citar is the recommended bibliographic frontend.

I'll see if I can play around in the code to get Citar supported, since Vertico and Citar are the defaults on Doom.

[Vidianos-Giannitsis]

Should be easy enough I think, as I haven't used too much code from bibtex completion. I assume there is an underlying function that finds the whole database of bibtex entries in Citar and after that, it will at most need some reformatting to find the nodes.

Thanks for your continued support!

On Mon, 28 Mar 2022, 23:07 Jay L. Colbert, @.***> wrote:

If using Vertico or Selectrum as completion interfaces, Citar https://github.com/bdarcus/citar is the recommended bibliographic frontend.

I'll see if I can play around in the code to get Citar supported, since Vertico and Citar are the defaults on Doom.

— Reply to this email directly, view it on GitHub https://github.com/Vidianos-Giannitsis/zetteldesk.el/issues/5, or unsubscribe https://github.com/notifications/unsubscribe-auth/APGY5W62PXOOVANRAZ2JHA3VCIGQTANCNFSM5R4HQQWQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[Vidianos-Giannitsis]

I looked a bit into Citar. There appears to be (citar--get-candidates) which returns a very similar looking list to that of (bibtex-completion-candidates), however I could not find something in that list that determines whether each entry has a note. The bibtex variant has the "=has-note=" string in each entry which can easily be used for this purpose.

However, there seems to be the function (citar-has-note) which returns a predicate function determining whether an entry has a note. Using this as (citar--completion-table (citar--get-candidates) (citar-has-note)) (which is the way citar-select-ref uses it) is a step in the right direction. However, the object returned, cannot be manipulated like how zetteldesk-node-ref-p does for example. It appears to be a list of closures and from my crude tests I couldn't find a way to get every bibtex entry with a note and convert that to a list of org-roam nodes.

If you pass it to completing-read as (completing-read "Refs: " (citar--completion-table (citar--get-candidates) (citar-has-note))) for example, will give a prompt with only the correct entries, so there's probably a way to get all nodes associated with such bibtex entries.

I hope this can aid you in finding the solution. With a bit more knowledge on how Citar works, I believe you should be able to handle it. If completing-read can get the list, its definitely possible to do it, I just had some trouble working with the result of (citar--completion-table (citar--get-candidates) (citar-has-note)). If you can manipulate that list and take the cite-key of every entry (zetteldesk-node-from-refs) should give the list of nodes, therefore making the rest of the package to work.

Let me know if you find something

[jlcolbert]

Oh excellent! Yeah, I had found the citar--get-candidates function, too, but I haven't had much time to play around with it this week. Writing my annual report for work and whatnot. This helps, though! I'll look into after I turn my report in tomorrow.

[Vidianos-Giannitsis]

Yeah, no worries. There is no real rush, you can check it out when you have time.

On Thu, 31 Mar 2022, 16:21 Jay L. Colbert, @.***> wrote:

Oh excellent! Yeah, I had found the citar--get-candidates function, too, but I haven't had much time to play around with it this week. Writing my annual report for work and whatnot. This helps, though! I'll look into after I turn my report in tomorrow.

— Reply to this email directly, view it on GitHub https://github.com/Vidianos-Giannitsis/zetteldesk.el/issues/5#issuecomment-1084569136, or unsubscribe https://github.com/notifications/unsubscribe-auth/APGY5W5UYJWXKQGE2HELNMDVCWRE3ANCNFSM5R4HQQWQ . You are receiving this because you commented.Message ID: @.***>

[Vidianos-Giannitsis]

Hey, have you by any means had any time to look more into this? I have been busy with the zetteldesk-remark extension and the documentation of zetteldesk-ref so I haven't looked more into it myself.

[jlcolbert]

I haven't yet, no. I'm gonna try to get to it soon. Doing my annual report really burnt me out. The Joys of Academia.

[bdarcus]

Citar author/maintainer here.

I don't really follow the details here, or what you ideally need. But feel free to ask or submit a PR over at that repo.

For example, if this is the ideal solution you need:

I could not find something in that list that determines whether each entry has a note. The bibtex variant has the "=has-note=" string in each entry which can easily be used for this purpose.

... it should be relatively painless to add (though I'm not certain how ATM).

Currently, "has:note" gets added to the actual candidate strings, as hidden text.

https://github.com/bdarcus/citar/blob/a2d7a9975a1535a8cc587ca1ed8847e743203da6/citar.el#L592 https://github.com/bdarcus/citar/blob/a2d7a9975a1535a8cc587ca1ed8847e743203da6/citar.el#L607

PS - might be helpful to add a screenshot or two to the README?

[Vidianos-Giannitsis]

Oh, glad to see you here mate.

So, the way I initially implemented it with bibtex-completion is this: First, I take all valid bibtex entries in my bibliography and add them in a list I filter that list to only include bibtex entries that have a note (which in ivy-bibtex is notes with a =has-note= field) From that list, I create a list of citekeys as that info is stored in the above list And from the list of citekeys, I get a list of nodes through org-roam-node-refs which I can then display with the various org-roam functions.

The relevant code is

(defun zetteldesk-ref-note-refs-p ()
  "Predicate function to find all bibtex completion candidates with a note.

Checks if every candidate has the \"=has-note=\" tag using
`assoc' and if it does, collects that candidate."
  (cl-loop for ref in (bibtex-completion-candidates)
       if (assoc "=has-note=" ref)
       collect ref))

(defun zetteldesk-ref-citekey-from-refs ()
  "Find the \"=key=\" tag from a list of candidates.

The list is collected with `zetteldesk-ref-note-refs-p' which is a
list of candidates that have notes. Collects it using `assoc'."
  (cl-loop for ref in (zetteldesk-ref-note-refs-p)
       collect (assoc "=key=" ref)))

(defun zetteldesk-ref-node-from-refs ()
  "Collects a list of ref nodes.

The nodes are collected from their citekey using
`org-roam-node-from-ref', while the citekeys themselves are
collected from `zetteldesk-ref-citekey-from-refs'."
  (cl-loop for ref in (zetteldesk-ref-citekey-from-refs)
       collect (org-roam-node-from-ref (concat "cite:" (cdr ref)))))

I assume it should be rather easy to get something like this with Citar, but I can't say I looked into it too much due to time shortage and not knowing much about Citar.

There's no screenshots in the README, because I thought I shouldn't include too much about the other extensions in the README but mainly focus on the main package and just give these a mention. If you check the wiki section of the extension there's (I think at least) ample info. I could reconsider this, but I thought this was more practical tbh.

If you have any other question I will be happy to answer it. Thanks for showing up, to help with this! Cheers

[bdarcus]

So you need a list keys for all entries that have associated notes?

E.g.:

("doe1" "doe2" "doe3")

That's all?

If so, could you not just take the code you reference above to filter the candidates, and from there extract the keys?

EDIT: maybe not. I'll dig a little.

[bdarcus]

OK, I've linked a simple PR.

With that, to get a list of keys:

(seq-map #'cadr 
  (seq-filter 
    (lambda (e) 
      (assoc "has-note" e)) 
  (citar--get-candidates)))

Would this help?

[Vidianos-Giannitsis]

This works exactly how I want it to, thanks for the very quick response!

Ok, just one more quick question, to fully port everything I have to citar. Besides this function, which takes citekeys and displays nodes with the org-roam UI, I have in the original file another one which shows the results in the Ivy-bibtex UI. From a brief look at the code, this should be fairly easy to do with citar, as it already has a function which opens the note of an entry and its backend has a filter function. But, what I need is a predicate that checks if an entry has a note AND is part of the zetteldesk. For doing that, I collect a list of citekeys of all zetteldesk ref-nodes.

So essentially, I need to add a predicate that checks if citar-has-note returns t, but also the citekey matches that of the list of nodes in the zetteldesk. That list is obtained with

(defun zetteldesk-ref-citekey-from-node ()
  "Collects the citekeys of org-roam-nodes in the `zetteldesk-desktop'.

Ignores nodes for which `org-roam-node-refs' returns nil."
  (let* ((init-list (org-roam-node-list))
     (zetteldesk-nodes (cl-remove-if-not #'zetteldesk-node-p init-list)))
    (cl-loop for node in zetteldesk-nodes
         if (org-roam-node-refs node)
         collect (car (org-roam-node-refs node)))))

I will try it myself when I have time, but I kinda need to go now so I wanted to give you an answer that this works and what else I want to look into. If you have the time to make such predicate, it would be very helpful for me however.

Thanks again for the help

[bdarcus]

If you have the time to make such predicate, it would be very helpful for me however.

I'm confused here. Wouldn't that predicate go here? Do you even need a predicate?

Seems to me you just need the intersection of two lists of keys; one from citar, and the other from zetteldesk:

(cl-intersection zd-citar-keys zd-keys :test #'equal)
(seq-intersection zd-citar-keys zd-keys)

In any case, there's a bit of conversation on the citar issue that has to do with some nuances. But I think I figured out another, more general, solution there that will also work; an optional predicate function to filter results from citar--get-candidates.

(seq-map #'cadr (citar--get-candidates t (citar-has-note)))

I've merged both commits, so there's now more than one option.

Edit: I'm currently cleaning up the API symbol names ahead of a 1.0 tag, so it's possible those details may change.

[Vidianos-Giannitsis]

Yeah, its for here I just thought since you know how Citar's predicate functions work, if you could give me some help with how this would be implemented. But its ok, I will figure it out, it doesn't look so hard, you have already done enough to help me out.

Sorry for not replying in the other thread, I checked it out, but I am super busy right now so I didn't respond as I wasn't aware of the exact technicalities you were referring to.

Anyways, I will look at both when I have the time and will answer. I just wanted to reply here as your responses have been really fast and I appreciate it. But in general this is shaping out to be a very hectic month for me so I don't want to spend too much time here. Thanks for everything you did however!

On 18/5/22 13:50, bdarcus wrote:

If you have the time to make such predicate, it would be very
helpful for me however.

I'm confused here. Wouldn't that predicate go here?

In any case, there's a bit of conversation on the citar issue that has to do with some nuances. But I think I figured out a solution there that will work, and will merge it soon.

— Reply to this email directly, view it on GitHub https://github.com/Vidianos-Giannitsis/zetteldesk.el/issues/5#issuecomment-1129859814, or unsubscribe https://github.com/notifications/unsubscribe-auth/APGY5WZRCZTAZLAOG4RSQJLVKTDQ3ANCNFSM5R4HQQWQ. You are receiving this because you commented.Message ID: @.***>

[Vidianos-Giannitsis]

Hey, I just saw your edit and honestly I was not even aware elisp had an intersection function. This is most likely to solve what I was looking for indeed. Thanks for the mention. I will work on this when I have time and push the changes to a seperate zetteldesk-ref-citar package. Again thank you, your help has been invaluable

[bdarcus]

You may be interested in this question I just asked Jethro; might overlap.

https://github.com/emacs-citar/citar/issues/518#issuecomment-1133216770

EDIT: that question evolved into this PR:

https://github.com/emacs-citar/citar/pull/603

Once I merge that, probably tomorrow (May 26), I will add a tiny little package for tighter org-roam v2 integration.

EDIT 2: I decided to move that to it's own repo:

https://github.com/emacs-citar/citar-org-roam/pulls

My expectation is it remains a very small package. I just thought it easier to have a repo with its own issue tracker, permissions, etc.

EDIT 3: here's what an adapted version of the body of your zetteldesk-ref-node-from-refs function might look like:

(cl-loop for cand in (citar--get-candidates nil (citar-has-note)) 
  for key = (cadr cand)
  collect (org-roam-node-from-ref (concat "@" key)))

PS - am I the only one that thinks it's a problem there are multiple syntaxes for ref keys here?

[bdarcus]

Just FYI, I'm currently working on some major architectural changes in citar, which I hope to merge in the next week or so:

https://github.com/emacs-citar/citar/pull/628

This will have implications for this discussion, as I will probably deprecate citar--get-candidates.

But I'll be thinking about these sorts of use cases as we finalize it, and if anything, it should be an improvement.

[Vidianos-Giannitsis]

Oh I see. I was actually planning on starting this, this weekend as I have free time again (yay), but if you have plans on changing Citar I will keep it on hold on I guess. Thanks for the heads up

[bdarcus]

If you have time, you can also weigh in on the PR and linked issue. Among the goals is it should work really well for this and similar projects.

~The most relevant change is a citar--notes-cache hash table, so you can do something like this:~

Nevermind; reverted that. The way things work in the main branch will stay, with the citar-has-note-functions variable.

Earlier I had a list of citekeys, but this branch restructures everything to use hash tables, where aside from the completion cache, all others use citekey as key.

[Vidianos-Giannitsis]

Sure, I will give it a closer look

[Vidianos-Giannitsis]

Ok I had a look at the PR and gave my opinion on at least what I understood of it. It does sound like a good change.

Btw since there is still discussion here and I haven't actually spent the time to write this extension fully, I'll reopen the issue

[bdarcus]

I'm still planning to release this, either separately, or as part of citar, in part to test to make the citar changes all work correctly!

https://github.com/emacs-citar/citar-org-roam/blob/initial-commit/citar-org-roam.el

But my intention too will be that other projects, like potentially this one, can use it.

[Vidianos-Giannitsis]

Yeah, this also looks great tbh as it has a close interaction with org-roam. Could definitely be helpful

[bdarcus]

Yeah, this also looks great tbh as it has a close interaction with org-roam. Could definitely be helpful

Here's citar-org-roam, with citar "has-note" indicators updating "live" (each time calling citar-select-ref).

This is on the new PR:

https://github.com/emacs-citar/citar/pull/634

[Vidianos-Giannitsis]

The above commit implements what I would consider the complete port of zetteldesk-ref to use citar as its backend, which seems to work as expected (produce the same results as the ivy-bibtex implementation) and is also significantly faster it seems, although that is probably because I wrote the filter on the other function with a loop which I doubt is optimal performance wise.

Thanks again for all the help you gave me in trying to implement this. If you have the time, feel free to check it out. Once the other extensions' PRs (which are already submitted) go through to MELPA I will probably upload this as well.

P.S. After working a bit with Citar and seeing how active it is as a project, I am seriously considering moving my bibliography setup to it :D. Although that is certain to take a while so I am not sure when.

[Vidianos-Giannitsis]

Closing this again for now as its currently functional. @bdarcus, when the changes are commited over to Citar do inform me to re-open this issue and fix the package to work again!

Thanks!

[bdarcus]

OK, I've merged https://github.com/emacs-citar/citar/pull/634, FYI.

Also, this mostly works with it:

https://github.com/emacs-citar/citar-org-roam/blob/main/citar-org-roam.el

Consider the note API slightly experimental, in the sense I still have a question or two about it, which I've noted in code comments.

[Vidianos-Giannitsis]

Ok, cool

Will pull down the latest changes and see what needs to be changed in zetteldesk-ref-citar. Will probably make use of citar-org-roam as well!

Thanks for the notification

On 4/7/22 16:49, bdarcus wrote:

OK, I've merged #634, FYI.

Also, this mostly works with it:

https://github.com/emacs-citar/citar-org-roam/blob/main/citar-org-roam.el

Consider the note API slightly experimental, in the sense I still have a question or two about it, which I've noted in code comments.

— Reply to this email directly, view it on GitHub https://github.com/Vidianos-Giannitsis/zetteldesk.el/issues/5#issuecomment-1173844760, or unsubscribe https://github.com/notifications/unsubscribe-auth/APGY5W2VUC376QC6DTJSLSTVSLTVTANCNFSM5R4HQQWQ. You are receiving this because you modified the open/close state.Message ID: @.***>

[bdarcus]

Let me know if you know how to fix https://github.com/emacs-citar/citar-org-roam/issues/2.