Closed bdarcus closed 3 years ago
In theory, citproc-el
automatically links the content of URL fields this way in its HTML and LaTeX output. Have you encountered different behaviour?
OK, great. But it doesn't work for dois currently; right? I get this in HTML export, without a link.
https://doi.org/10.1525/aa.2001.103.1.45
No, dois are not linked in HTML output.
OK, revised the title and description to narrow it.
The doi issue in general threw me for a loop, and I just noticed this while trying to figure that out.
What's odd is you have a doi in regular text, org-mode will export it linked in html.
BTW, I noticed there's this, but no equivalent for CSL.
(defcustom org-cite-biblatex-options nil
EDIT:
I'd love to submit a PR to change this behaviour but haven't been able to understand the code well enough to instrument it in an efficient way. Here's what I've tried so far:
rendered-var URL
pattern at lines 95,159,225 in bibtex-formatters.el. This seems to have no effect on the DOIs, s o I'm clearly missing something. ("doi" . DOI)
to citeproc-bt--to-csl-keys-alist
in citeproc-bibtex.el -- that seems to work, actually, and is probably worth doing, right? Betterbibtex (Zotero exporter) uses all lowercase key names, so the doi
field is otherwise lost in that use case. also: not sure if this deserves its own issue, but I would love to move the URL from visible text in my style to a simple <a>
link around the title, as described in this comment on citeproc's issue 76, referenced above. Presumably this would be accomplished with some kind of (rendered-var . title)
lambda in bibtex-formatters, but I'm not quite sure how to do it in the existing citeproc.el structure, which uses a bunch of more advanced lisp stuff that makes my head hurt.
As in #25? See, in particular: https://github.com/andras-simonyi/citeproc-el/issues/25#issuecomment-855225435.
As in #25? See, in particular: #25 (comment).
huh, actually no, though that is super cool. I want instead something like
McNeil, Donald. 2011. "<a` href="https://foo.org/bar">Fine Grain, Global City...</a>." etc. etc. http://dx.doi.org/1023812.123.3423
which is e.g. how Zotpress manages linking, if I'm understanding that citeproc issue properly.
@andras-simonyi any pointers on how I might try to amend the code to make DOI linkable? And is this something you would support?
Could you specify the required behavior? If the values of the DOI and PMID fields are [doi] and [pmid] then what should the link targets be exactly? https://doi.org/[doi] for DOI? And what about PMIDs? Or should we make these mappings configurable?
Here's the PR for the appendix we added to the CSL spec. Is this helpful?
https://github.com/citation-style-language/documentation/pull/86
I think there is some complexity here as the style itself may add the http://
prefix. e.g., here is apa.csl:
<macro name="access">
<choose>
<if variable="DOI" match="any">
<text variable="DOI" prefix="https://doi.org/"/>
</if>
.......
</choose>
</macro>
I personally would prefer to have a defcustom for this, e.g.
(defcustom citeproc-doi-prefix "https://doi.org/"
"URL prefix for DOI fields in exported bibliography entries")
and then on export replace the DOI with
<a href="{citeproc-doi-prefix}{DOI}">{DOI}</a>
or similar.
Does that sound right to you, Bruce? Do you know if this will screw things up with styles such as APA?
(also, I don't really work with PMID so have nothing to say about that unfortunately).
I think there is some complexity here as the style itself may add the http:// prefix. e.g., here is apa.csl:
Isn't that orthogonal to actual linking though?
That code determines what will be rendered; not what the link target is.
The PR, OTOH, mostly focuses on what the target should be.
Does that sound right to you, Bruce? Do you know if this will screw things up with styles such as APA?
I'm not really sure.
The idea of that PR was let's see how far we can get with some basic rules, implemented in processors, with the expectation it would cover most use cases.
I'd like see what @andras-simonyi thinks about this. There's always room to iterate those recommendations.
I think there is some complexity here as the style itself may add the http:// prefix. e.g., here is apa.csl:
Isn't that orthogonal to actual linking though?
That code determines what will be rendered; not what the link target is.
I don't really know enough about where citeproc.el interfaces with the stylesheet. From what I can tell, the lambda functions in citeproc-fmt--html-alist
are responsible for that, and I imagine they would have to transform the field before the stylesheet acts on it. If that's the case, then the instruction
<text variable="DOI" prefix="https://doi.org/"/>
would potentially end up producing a string like https://doi.org<a href="https://doi.org/XXX.XXX.XX>https://doi.org/XXX.XXX.XX<a/>
Is this a legitimate worry?
I would test it out but there's something I clearly don't understand, as the modifications I have tried to make to citeproc-formatters.el
seem to have no effect.
I don't know this code, but from a CSL perspective, the snippet you show above just says to output DOIs as:
https://doi.org/XXX.XXX.XX
The new spec recommendation suggests doing this in HTML:
<a href="https://doi.org/XXX.XXX.XX">https://doi.org/XXX.XXX.XX</a>
So for sake of argument, if the above CSL fragment did not have a prefix, the result would be:
<a href="https://doi.org/XXX.XXX.XX">XXX.XXX.XX</a>
Thanks for all the comments, they are very useful! I've created a new branch 22-add_links for this feature and already added linking rendered DOIs, PMIDs and PMCIDs. Could you perhaps test whether the code does what it should? (It might also help in understanding the formatter "framework".) What is missing is to link the title if no explicit URL etc is present -- I intend to add that in the next 1-2 days.
Could you perhaps test whether the code does what it should?
As you know, I've had some issues getting these feature branches working together as I expect (for example, dates still go missing with json).
I tried to test this branch, but the output isn't including any dois, which maybe means I have something wrong with my data files, that I don't see ATM. Or some issue with the setup.
The code looks right though.
WDYT?
Oh wait, oddly, it works if I use bib files.
Which turns up an issue. This is what I get now:
https://doi.org/
<a href="https://doi.org/10.1111/j.1468-2427.2010.01027.x">10.1111/j.1468-2427.2010.01027.x</a>
I think it should be:
<a href="https://doi.org/10.1111/j.1468-2427.2010.01027.x">https://doi.org/10.1111/j.1468-2427.2010.01027.x</a>
Right? You're putting the link-prefix before the link ATM.
Edit: now I'm not sure sure. Does it depend where that prefix is coming from?
I also had a little trouble testing -- I'm not sure exactly why, but I think maybe because the lexical binding of citeproc-fmt--formatters-alist
means that its value doesn't actually update when the library is reloaded.? In any case, I wish that it were possible to create the formatters dynamically in case the value of the citeproc-fmt--html-alist
or citeproc-fmt--latex-alist
changes after the library is loaded, e.g., perhaps in a let-binding during export. .
Anyway, this is what i get (similar to Bruce):
<div class="csl-entry"><a name="citeproc_bib_item_4"></a>Zubaida, S. (2009). The idea of “Indian food”, between the colonial and the global. <i>Food & History</i>, <i>7</i>(1), 191–209. https://doi.org/<a href="10.1484/j.food.1.100643">10.1484/j.food.1.100643</a></div>
Bruce, I don't think the issue here is with citeproc-el, but with CSL itself -- the style returns a text string with https://doi.org/
prepended to the DOI. citeproc-el can only modify the value of the DOI itself, if I understand correctly. So to get the output you wnat (and which is obviously preferable) you'd either have to modify the style or do a regexp search-and-replace on the rendered citation itself. I suck at regex so that would likely take me many hours; a partial solution that works for doi links at least is:
(defun citeproc-fmt--html-bib-formatter (items _bib-format)
"Return a html bibliography from already formatted ITEMS."
(concat "<div class=\"csl-bib-body\">\n"
(mapconcat (lambda (i)
(concat " <div class=\"csl-entry\">"
(replace-regexp-in-string
"\\(https://doi.org/\\)\\(<a href=\".+?\">\\)\\(.+?\\)\\(</a>\\)"
"\\2\\1\\3\\4"
i )
"</div>\n"))
items
"")
"</div>"))
Is there a better solution though?
freaking emacs regexp backslash tax is killing me, gotta say.
how about this, actually:
(defun citeproc-fmt--html-bib-formatter (items _bib-format)
"Return a html bibliography from already formatted ITEMS."
(concat "<div class=\"csl-bib-body\">\n"
(mapconcat (lambda (i)
(concat " <div class=\"csl-entry\">"
(replace-regexp-in-string
"\\(https://doi.org/\\|http://www.ncbi.nlm.nih.gov/pubmed/\\|http://www.ncbi.nlm.nih.gov/pmc/articles/\\)\\(<a href=\".+?\">\\)\\(.+?\\)\\(</a>\\)"
;;"\\(https://doi.org/\\)\\(<a href=\".+?\">\\)\\(.+?\\)\\(</a>\\)"
"\\2\\1\\3\\4"
i )
"</div>\n"))
items
"")
"</div>"))
It covers all the URL prefixes I found in the csl styles repository. It doesn't link the PMID:
, or PMCID
prefixes in some journal formats. I think this may be a weakness of the new CSL recommendation, or anyway an indication that it's incompatible with some styles? What do you think, Bruce?
Edit: still feels pretty ad hoc, though.
Bruce, I don't think the issue here is with citeproc-el, but with CSL itself -- the style returns a text string with
https://doi.org/
prepended to the DOI.
I'm actually still not sure on what is responsible how; only that the output it not ideal.
I'm actually still not sure on what is responsible how; only that the output it not ideal.
Did you try my fix above?
Did you try my fix above?
No, I didn't.
It covers all the URL prefixes I found in the csl styles repository. It doesn't link the PMID:, or PMCID prefixes in some journal formats. I think this may be a weakness of the new CSL recommendation, or anyway an indication that it's incompatible with some styles?
Well, like I said, it was a first cut at a recommendation. So it could be that this experience suggests revisions.
But to go back to the basic CSL -> output logic.
Let's assume this CSL:
<text variable="DOI" prefix="doi: "/>
<text variable="DOI" prefix="https://doi.org/"/>
So we have one of two choices:
<a href="https://doi.org/XXX.XXX.XX">doi: XXX.XXX.XX</a>
.Right?
So we have one of two choices:
1. say the prefix is included in the link, which would mean in Case 1 `<a href="https://doi.org/XXX.XXX.XX">doi: XXX.XXX.XX</a>`. 2. say the prefix precedes the link, unless the prefix preceding the link is in fact a base URL, in which case do 1
Right?
Well, I think both you and Andras will understand this a lot better than me, but as I understand it, the various functions in citeproc-fmt--html-alist
are only able to act directly on the underlying values in the dataset. The CSL transformations themselves rae out of scope; when we have something like
<text value="doi:"/>
<text variable="DOI"/>
citeproc-el has no way to know about the "doi:" in advance. So, the only way to fix it is some post-hoc manipulation, I think.
though maybe I misunderstand. Working my way thorugh citeproc-rt right now.
It's a bit beyond me but it seems like the solution might be in citeproc-rt-render-affixes
(citeproc-rt, line 413). I'm not sure it's entirely simple to adjust. Lines 437-442 or so look particularly relevant.
I wish the lambda functions in citeproc-fmt--html-alist
had access to the whole citation object somehow. But maybe they do and I just don't know.
Thanks again for the comments! @titaniumbones: just a short explanation of what is going on code-wise: as you have seen, the library uses its own internal rich text representation, and, ideally, formatters should act only as (relatively) simple converters to produce the actual outputs in the required formats. The formatter API has two layers: a low-level one, which is very general: the only assumption made about the RT component of a citeproc-formatter
struct is that it maps rich texts to their properly formatted version, and this function is called with the whole rich text formatted bibliography entry. The other, "convenience layer" of the API provides some functions that -- hopefully -- make creating this function easier, because they require only a mapping from rich text attribute-value pairs to corresponding text manipulation functions.
Because of the above setup, it seems to me that, ideally, linking should happen already at the internal rich text representation level, so I will try to change the code this way.
As for the trouble with prefixes that add URI components to some variable values: I see one potential problem with the recommendation as it is. If the URI resulting from adding the prefix is different from the recommended target then we end up linking an URI anchor to a different target which is rather confusing IMHO.
I'll look forward to seeing what you produce -- hopefully once you have an elegant solution I will be able to study it to understand a little better what is going on.
As I have worked on this I realize that one end result I would very much like to be able to produce is an informational tooltip or title attribute that can be accessed by hovering over the citation, much as oc-basic's activate
echoes a semi-formatted citation to the minibuffer. It seems clear to me that that should not be citeproc's job, but I wonder if there's anything in the architecture that can be leveraged by a processor (ox-csl
or something derived from it) to make such features easier to implement. For instance,is there a finishing stage at which the citation can get access to the final bibliography entr[y|ies] and potentially build a tooltip out of them; or could the parsed bibliography entry text just be joined by spaces to create a title
attribute on the citatlion link? Etc.
I am shocked that we have finally gotten to this stage after so many years of half-baked solutions. I think it will revolutionize my writing, actually.
As I have worked on this I realize that one end result I would very much like to be able to produce is an informational tooltip or title attribute that can be accessed by hovering over the citation, much as oc-basic's
activate
echoes a semi-formatted citation to the minibuffer.
https://github.com/andras-simonyi/org-cite-csl-activate
If you hover on a citation, you will get a pop-up with a mini-bibliography.
There was talk of including that functionality in oc-csl
; not sure what the status of that is.
I see one potential problem with the recommendation as it is. If the URI resulting from adding the prefix is different from the recommended target then we end up linking an URI anchor to a different target which is rather confusing IMHO.
If you have suggestions for amendments for that language based on this work, please let me know. Maybe best would be a PR, or an issue here:
I think I've implemented (still in the feature branch) everything required by Appendix VI -- I'd appreciate if you could try it out and comment. Thanks!
I did a quick test with chicago-note and apa and looks good to me @andras-simonyi.
Thanks.
So do you conclude we need to amend that recommendation?
it looks awesome to me.
I haven't looked through the code yet, but am I right that there is supposed to be a case in which the URL gets wrapped around the title? Do you know of a style I should try out in order to test this?
On 7/7/21 11:22 AM, bdarcus wrote:
EXTERNAL EMAIL:
I did a quick test with chicago-note and apa and looks good to me @andras-simonyi https://github.com/andras-simonyi.
Thanks.
So do you conclude we need to amend that recommendation?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/andras-simonyi/citeproc-el/issues/22#issuecomment-875697296, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACY6NHIC626WRQHH5T5ENDTWRWL3ANCNFSM4522CSLA.
I haven't looked through the code yet, but am I right that there is supposed to be a case in which the URL gets wrapped around the title? Do you know of a style I should try out in order to test this?
Yes, I managed to test it with the apa-no-doi-no-issue
style in the repository.
@bdarcus wrote:
So do you conclude we need to amend that recommendation?
Well, looking at some of the styles it's difficult to tell. I find it problematic to link a well formed URL string as anchor to a different URL but it appears to be the safest option. Some of the styles I've seen add URL prefixes to DOIs which don't make much sense IIRC.
Meanwhile I've merged the associated feature-branch into master, so planning to close this issue soonish.
Another non-essential-nice-to-have feature to consider.
See this commit.
The option would just allow this in HTML export:
... or this:
.... to be:
... or: