RiegerSven
commented
3 years ago same problem here.
Closed jrennstich closed 2 years ago
RiegerSven
commented
3 years ago same problem here.
crsh
commented
3 years ago Thanks, I'll look into it.
psychtek
commented
3 years ago Getting the same error as above with a fresh Ubuntu install:
> sessioninfo::session_info()
─ Session info ─────────────────────────────────────────────────────────────────────────────────
setting value
version R version 4.0.3 (2020-10-10)
os Ubuntu 20.04.1 LTS
system x86_64, linux-gnu
ui RStudio
language en_AU:en
collate en_AU.UTF-8
ctype en_AU.UTF-8
tz Australia/Melbourne
date 2020-11-01
─ Packages ─────────────────────────────────────────────────────────────────────────────────────
package * version date lib source
assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.0.3)
bookdown 0.21 2020-10-13 [1] CRAN (R 4.0.3)
cli 2.1.0 2020-10-12 [1] CRAN (R 4.0.3)
crayon 1.3.4 2017-09-16 [1] CRAN (R 4.0.3)
digest 0.6.27 2020-10-24 [1] CRAN (R 4.0.3)
evaluate 0.14 2019-05-28 [1] CRAN (R 4.0.3)
fansi 0.4.1 2020-01-08 [1] CRAN (R 4.0.3)
glue 1.4.2 2020-08-27 [1] CRAN (R 4.0.3)
htmltools 0.5.0 2020-06-16 [1] CRAN (R 4.0.3)
knitr 1.30 2020-09-22 [1] CRAN (R 4.0.3)
papaja 0.1.0.9997 2020-10-27 [1] Github (crsh/papaja@c6648ad)
rlang 0.4.8 2020-10-08 [1] CRAN (R 4.0.3)
rmarkdown 2.5 2020-10-21 [1] CRAN (R 4.0.3)
rstudioapi 0.11 2020-02-07 [1] CRAN (R 4.0.3)
sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.0.3)
tinytex 0.26 2020-09-22 [1] CRAN (R 4.0.3)
withr 2.3.0 2020-09-22 [1] CRAN (R 4.0.3)
xfun 0.18 2020-09-29 [1] CRAN (R 4.0.3)
yaml 2.2.1 2020-02-01 [1] CRAN (R 4.0.3)
[1] /home/psychtek/R/x86_64-pc-linux-gnu-library/4.0
[2] /usr/local/lib/R/site-library
[3] /usr/lib/R/site-library
[4] /usr/lib/R/library> rmarkdown::pandoc_version()
[1] ‘2.11.0.2’Just to give a quick update on this: I'm pretty sure this is a problem caused by the fact that the new pandoc version replaces pandoc-citeproc with citeproc. As far as I understand from https://github.com/jgm/pandoc-citeproc/issues/448#issuecomment-627697558 the pandoc-citeproc version distributed with the RStudio preview release may be compiled for the wrong version of pandoc. If you would like to speed up a fix for this issue, you could help me by trying to manually call pandoc-citeproc in a regular pdf_document to see if that gives the same error. I think the following should do the trick:
output:
pdf_document:
pandoc_args: [
"--filter", "path/to/pandoc-citeproc"
]You can find the path to pandoc-citeproc using rmarkdown:::pandoc_citeproc().
jrennstich
commented
3 years ago Just to give a quick update on this: I'm pretty sure this is a problem caused by the fact that the new
pandocversion replacespandoc-citeprocwithciteproc. As far as I understand from jgm/pandoc-citeproc#448 (comment) thepandoc-citeprocversion distributed with the RStudio preview release may be compiled for the wrong version ofpandoc. If you would like to speed up a fix for this issue, you could help me by trying to manually callpandoc-citeprocin a regularpdf_documentto see if that gives the same error. I think the following should do the trick:output: pdf_document: pandoc_args: [ "--filter", "path/to/pandoc-citeproc" ]You can find the path to
pandoc-citeprocusingrmarkdown:::pandoc_citeproc().
Do you mean using the papaja template? or just any regular markdown file?
I can compile PDFs using other markdown files or komaletter for example.
crsh
commented
3 years ago What I mean is, try to take a vanilla pdf_document RMarkdown document and replace output: pdf_document by the code above. If this reproduces the error, I think the error may be due to an incorrectly compiled pandoc-citeproc, as suggested in the issue I linked to.
jrennstich
commented
3 years ago Test2.pdf Works w/o problems on my end. Get the attached resulting PDF
Code used:
---
output:
pdf_document: default
html_document: default
---
# Headline
Some text that would appear here so there's something to render.> sessioninfo::session_info()
─ Session info ───────────────────────────────────────────────────────────────────────────────────────────────────────────
setting value
version R version 4.0.3 (2020-10-10)
os macOS Big Sur 10.16
system x86_64, darwin17.0
ui RStudio
language (EN)
collate en_US.UTF-8
ctype en_US.UTF-8
tz Europe/Berlin
date 2020-11-17
─ Packages ───────────────────────────────────────────────────────────────────────────────────────────────────────────────
package * version date lib source
assertthat 0.2.1 2019-03-21 [2] CRAN (R 4.0.0)
cli 2.1.0 2020-10-12 [2] CRAN (R 4.0.2)
crayon 1.3.4 2017-09-16 [2] CRAN (R 4.0.0)
digest 0.6.27 2020-10-24 [2] CRAN (R 4.0.2)
evaluate 0.14 2019-05-28 [2] CRAN (R 4.0.0)
fansi 0.4.1 2020-01-08 [2] CRAN (R 4.0.0)
glue 1.4.2 2020-08-27 [2] CRAN (R 4.0.2)
htmltools 0.5.0.9001 2020-10-22 [1] Github (rstudio/htmltools@5d42d84)
jsonlite 1.7.1 2020-09-07 [2] CRAN (R 4.0.2)
knitr 1.30 2020-09-22 [2] CRAN (R 4.0.2)
lattice 0.20-41 2020-04-02 [2] CRAN (R 4.0.3)
Matrix 1.2-18 2019-11-27 [2] CRAN (R 4.0.3)
Rcpp 1.0.5 2020-07-06 [2] CRAN (R 4.0.0)
reticulate 1.18 2020-10-25 [1] CRAN (R 4.0.2)
rlang 0.4.8 2020-10-08 [2] CRAN (R 4.0.2)
rmarkdown 2.5 2020-10-21 [2] CRAN (R 4.0.3)
rsconnect 0.8.16 2019-12-13 [2] CRAN (R 4.0.0)
rstudioapi 0.13 2020-11-12 [2] CRAN (R 4.0.3)
sessioninfo 1.1.1 2018-11-05 [2] CRAN (R 4.0.0)
withr 2.3.0 2020-09-22 [2] CRAN (R 4.0.2)
xfun 0.19 2020-10-30 [1] CRAN (R 4.0.3)
yaml 2.2.1 2020-02-01 [2] CRAN (R 4.0.0) As per https://github.com/crsh/papaja/issues/427#issuecomment-728805035, an informative test would be to use the following code:
---
output:
pdf_document:
pandoc_args: [
"--filter", "path/to/pandoc-citeproc"
]
---
# Headline
Some text that would appear here so there's something to render.and replace path/to/pandoc-citeproc by the output of rmarkdown:::pandoc_citeproc().
jrennstich
commented
3 years ago Sorry, my bad, I did that now and got the following result. (Multitasking w student communication is not a good idea in such cases).
output file: Test.knit.md
[WARNING] Deprecated: pandoc-citeproc filter. Use --citeproc instead.
Error running filter path/to/pandoc-citeproc:
Could not find executable path/to/pandoc-citeproc
Error: pandoc document conversion failed with error 83
Execution haltedI ran rmarkdown:::pandoc_citeproc() and the output was [1] "pandoc-citeproc" .
and once I replaced it
Error running filter pandoc_citeproc:
Could not find executable pandoc_citeproc
Error: pandoc document conversion failed with error 83
Execution haltedreplacing it with --citeproc as per output of error message (I also tried pandoc--citeproc) yields same result
Error running filter --citeproc:
Could not find executable --citeproc
Error: pandoc document conversion failed with error 83
Execution haltedSorry, my bad, I did that now and got the following result. (Multitasking w student communication is not a good idea in such cases).
output file: Test.knit.md [WARNING] Deprecated: pandoc-citeproc filter. Use --citeproc instead. Error running filter path/to/pandoc-citeproc: Could not find executable path/to/pandoc-citeproc Error: pandoc document conversion failed with error 83 Execution haltedI ran
rmarkdown:::pandoc_citeproc()and the output was[1] "pandoc-citeproc". and once I replaced itError running filter pandoc_citeproc: Could not find executable pandoc_citeproc Error: pandoc document conversion failed with error 83 Execution haltedreplacing it with
--citeprocas per output of error message (I also triedpandoc--citeproc) yields same resultError running filter --citeproc: Could not find executable --citeproc Error: pandoc document conversion failed with error 83 Execution halted
My apologies if I don't really understand whether I have replaced the proper output from rmarkdown:::pandoc_citeproc()
crsh
commented
3 years ago Thanks for your efforts!
I'm not sure either, but based on your output, I would hope that the following should work:
---
output:
pdf_document:
pandoc_args: [
"--filter", "pandoc-citeproc"
]
---
# Headline
Some text that would appear here so there's something to render.Unfortunately, no :-(
[WARNING] Deprecated: pandoc-citeproc filter. Use --citeproc instead.
pandoc-citeproc: Error in $: Incompatible API versions: encoded with [1,22] but attempted to decode with [1,21].
CallStack (from HasCallStack):
error, called at ./Text/Pandoc/JSON.hs:112:48 in pandoc-types-1.21-LpPdBV1sPudK4kVghsVxWs:Text.Pandoc.JSON
Error running filter pandoc-citeproc:
Filter returned error status 1
Error: pandoc document conversion failed with error 83
Execution haltedI understand that this result may be frustrating to you, but I'm delighted about it. It means that the problem is not with papaja per se (although I need to update the package to make it compatible with citeproc), but with the pandoc-citeproc binary distributed with the RStudio preview release. I'll try to pass this issue along tonight.
cderv
commented
3 years ago Just shiming in as someone pinged up the unanswered issue here: https://community.rstudio.com/t/papaja-rmd-knit-problem-with-error-pandoc-document-conversion-failed-with-error-83/86818
Some context:
pandoc-citeproc external filter to use an internal --cireproc option. This changes must be handle in all tools that uses pandoc. --filter pandoc-citeproc to Pandoc 2.11 so do not add anything in the YAML file. This would work ok with Rmarkdown 2.5 and Pandoc 2.11
---
title: 'citation in footnote'
output: pdf_document
bibliography: packages.bib
---
```{r, include=FALSE}
knitr::write_bib("knitr", file = "packages.bib")See this citation [@R-knitr]
citeproc option is passed to pandoc command line only if a bibliography is used. You get the error with the example above using
```yaml
output:
pdf_document:
pandoc_args: [
"--filter", "pandoc-citeproc"
]because you are forcing an argument with Pandoc 2.11 that has been deprecated. Hence the message.
I believe you need to make papaja work with new pandoc 2.11. You are using https://github.com/crsh/papaja/blob/1f124807b6acad09aaae156f0fb9d59590e6650c/R/apa6_formats.R#L369 and this shouldn't be done with Pandoc 2.11 as the function for rmdfilter only consider the external filter, which the new pandoc version does not use.
I am not sure why you are doing that - you should let Rmarkdown adds the correct flag value depending on the Pandoc version. If you want do deal with that in your package, you need to check the Pandoc version and add the correct flag.
Hope it helps
crsh
commented
3 years ago Hi @cderv, thanks for comment! All of this makes sense and I was aware of the need to update to citeproc but I must be missunderstanding something. I'm puzzled by the fact that the use of an operational external filter would cause an error. The output we are seeing indicates that the filter is deprecated rather than defunct and it is a warning rather than an error.
[WARNING] Deprecated: pandoc-citeproc filter. Use --citeproc instead.It was my understanding that the error we are seeing is related to a filter being compiled against the wrong version of pandoc (jgm/pandoc-citeproc#448 (comment)).
pandoc-citeproc: Error in $: Incompatible API versions: encoded with [1,22] but attempted to decode with [1,21].
CallStack (from HasCallStack):
error, called at ./Text/Pandoc/JSON.hs:112:48 in pandoc-types-1.21-LpPdBV1sPudK4kVghsVxWs:Text.Pandoc.JSON
Error running filter pandoc-citeproc:
Filter returned error status 1
Error: pandoc document conversion failed with error 83
Execution haltedEither way, the issue will be fixed in rmdfiltr shortly.
jrennstich
commented
3 years ago Either way, the issue will be fixed in
rmdfiltrshortly.
Thx so much! I love papaja for its simplicity and versatility and really depend on it for my workflow. So this is great news!
cderv
commented
3 years ago @crsh I don't know about this warning for sure. I just know Pandoc now recommends to use --citeproc with Panodc 2.11 and above, and also that pandoc-citeproc filter is no more build with Pandoc from 2.11. That means also you need to build it yourself I guess as pandoc-citeproc and pandoc must be built together.
Re. operational filters and error, I know that filters depends on the AST so any change in the AST of Pandoc can cause previous operational external filters to fail with a newer version of Pandoc. Same, some filters operational for a current version of Pandoc may not work for an older version if it uses new stuff in the AST.
Anyway, glad you were able to fix it. It was interesting to see a case where one don't rely on rmarkdown choice and needed to recode the logic. It means some low level feature should be more easily accessible.
mariusbarth
commented
3 years ago Hi @jrennstich, I think we fixed the problem in the rmdfiltr package. It would be great if you could install the rmdfiltr package from github via remotes::install_github("crsh/rmdfiltr") and verify that it also fixes the problem with your specific setup.
jrennstich
commented
3 years ago Great, thanks for updating this! I installed the rmdfiltr and got the following error:
[WARNING] Citeproc: citation fruchtel-sozialer-2013b not found
Error running filter /Users/jrennstich/Library/R/4.0/library/rmdfiltr/replace_ampersands.lua:
...ch/Library/R/4.0/library/rmdfiltr/replace_ampersands.lua:27: attempt to index a nil value (field '?')
stack traceback:
...ch/Library/R/4.0/library/rmdfiltr/replace_ampersands.lua:27: in function 'replace_ampersands'
Error: pandoc document conversion failed with error 83
Execution haltedWith other papaja documents it works, however. So it seems to be this ampersand issue?
mariusbarth
commented
3 years ago Good to hear that most of your documents work with the updated rmdfiltr package. Concerning your new error, it seems to me this is another issue that might be caused by an inline citation that is not available, but I cannot reproduce the issue. Maybe @crsh has an idea what might cause this problem.
jrennstich
commented
3 years ago Thanks again @mariusbarth for solving this!
it seems to me this is another issue that might be caused by an inline citation that is not available
Unfortunately, those two error messages are unrelated I think. I continue to get the the ampersands.lua error with other files that have no missing citation.
crsh
commented
3 years ago Hi Joachim, sorry for the inconvenience. Would you be willing to share the problematic bibliography entry from the Bib(La)TeX-file with us for debuggin purposes?
jrennstich
commented
3 years ago Sure thing.
@book{fruchtel-sozialer-2013a,
title = {Sozialer Raum und Soziale Arbeit Fieldbook: Methoden und Techniken},
author = {Früchtel, Frank and Budde, Wolfgang and Cyprian, Gudrun},
date = {2013},
edition = {3},
publisher = {{Springer VS}},
location = {{Wiesbaden, Germany}},
annotation = {OCLC: 811587862},
isbn = {978-3-531-19047-1 978-3-531-18433-3},
keywords = {methodology,social,social-work,space,textbook},
langid = {german}
}The "2013b" was indeed simply missing.
mariusbarth
commented
3 years ago @crsh I tried to reproduce with pandoc 2.11 and RStudio 1.3.1093, but was not successful. So maybe it has sth to do with the RStudio pre-release
crsh
commented
3 years ago I was also unable to reproduce the problem with the current RStudio preview release (1.4.1056) and the GitHub-version of rmdfiltr:
---
title : "The title"
output : papaja::apa6_pdf
bibliography : bib.bib
---
```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```
@fruchtel-sozialer-2013a
# References@jrennstich, are there any other citations in you document? Can you easily share the full R Markdown and bibliography file for debugging purposes?
jrennstich
commented
3 years ago Ok, it seems I have narrowed down the problem somewhat. When I use the same reference
@book{wright-glut-2007,
title = {Glut: Mastering Information through the Ages},
publisher = {{Joseph Henry Press}},
author = {Alex Wright},
address = {{Washington, DC}},
year = {2007},
}
@book{menninger-zahlwort-1958,
title = {Zahlwort und Ziffer: eine Kulturgeschichte der Zahl},
edition = {2},
publisher = {{Vandenhoeck \& Ruprecht}},
author = {Karl Menninger},
address = {{G\"ottingen}},
year = {1958},
}in the following text:
Some text [^1].
[^1]: Some text with two references [@wright-glut-2007; @menninger-zahlwort-1958].it works fine. But the same reference alone
Some text [^1].
[^1]: Some text with two references [@wright-glut-2007].causes an error:
Error running filter /Users/jrennstich/Library/R/4.0/library/rmdfiltr/replace_ampersands.lua:
...ch/Library/R/4.0/library/rmdfiltr/replace_ampersands.lua:27: attempt to index a nil value (field '?')
stack traceback:
...ch/Library/R/4.0/library/rmdfiltr/replace_ampersands.lua:27: in function 'replace_ampersands'
Error: pandoc document conversion failed with error 83
In addition: Warning message:
In readLines(con = input_connection) :
incomplete final line found on '/Users/jrennstich/Dropbox/Digitalkompetenz/Test2.Rmd'
Execution haltedThat's independent of the reference, I tried several. It's always an issue when you use them singularly.
crsh
commented
3 years ago Excellent, thank you. I can reproduce the problem and will try to fix it as soon as possible.
crsh
commented
3 years ago Okay, so I now understand why the filter fails. I used the following document to compare the AST of a regular citation and another one in a footnote (it doesn't matter whether they are in text or not):
---
bibliography : bib.bib
---
@fruchtel-sozialer-2013a
Some text [^1].
[^1]: Some text with two references @fruchtel-sozialer-2013a.I used the following filter to print the information in the AST for each.
function debug_filter (el)
if #el.citations > 0 then
for key, value in pairs(el) do
print(key, " -- ", value)
end
for key, value in pairs(el.content) do
for key2,value2 in pairs(value) do
print(key2, " -- ", value2)
end
end
print(#el.citations)
for key, value in pairs(el.citations[1]) do
print(key, " -- ", value)
end
else
for key, value in pairs(el) do
print(key, " -- ", value)
end
for key, value in pairs(el.content) do
for key2,value2 in pairs(value) do
print(key2, " -- ", value2)
end
end
print(#el.citations)
end
return el
end
return {{Cite = debug_filter}}The in-text Cite-elment has two parts, citations, which contains something like metadata, and content, which contains the printed text. content is the same for both citations:
text -- Früchtel,
text -- Budde,
text -- &
text -- Cyprian
text -- (2013)citations is nil for the footnote citation. The filter, however, expects the same metadata as for the in-text citation:
prefix -- table: 0x7f8036407980
id -- fruchtel-sozialer-2013a
mode -- AuthorInText
note_num -- 1
hash -- 0
suffix -- table: 0x7f80364079c0This information is needed to determine whether a citation is AuthorInText and requires replacing &. As a quick fix, I will catch this exception in the filter, which generally also increase its robustness. I am, however, skeptical, whether the AST is indeed supposed to be missing the citations information. I will raise this issue with the citeproc developers.
jrennstich
commented
3 years ago Awesome, thanks for fixing this so quickly!
crsh
commented
3 years ago The issue has been fixed in pandoc and the new version of rmdfiltr has been accepted to CRAN---binaries will be available shortly.
AliceeDiaz
commented
3 years ago HI, sorry to jump into this discussion. This threat had been really helpful ti solve the knitting issue with remotes::install_github("crsh/rmdfiltr"). HOwever I am still having problems with papaja/citeproc not recognicing references that are certainly in the bib doc. It is not even finding the own Rreferences (e.g. [WARNING] Citeproc: citation R-base not found)The issue persists even when manually inserting the references throught the addings button. The number of unfound references decreases when knitting a htlm doc. references
I am using RStudio Cloud at the minute:
`─ Session info
setting value
version R version 4.0.3 (2020-10-10)
os Ubuntu 16.04.7 LTS
system x86_64, linux-gnu
ui RStudio
language (EN)
collate C.UTF-8
ctype C.UTF-8
tz Etc/UTC
date 2021-02-26
─ Packages ─── package * version date lib source
htmltools 0.5.1.1 2021-01-22 [1] RSPM (R 4.0.2)
httpuv 1.5.4 2020-06-06 [1] RSPM (R 4.0.2)
httr 1.4.2 2020-07-20 [1] RSPM (R 4.0.2)
jsonlite 1.7.2 2020-12-09 [1] RSPM (R 4.0.2)
papaja * 0.1.0.9997 2020-08-06 [1] Github (crsh/papaja@174aafa)
pillar 1.4.6 2020-07-10 [1] RSPM (R 4.0.2)
rmarkdown 2.7 2021-02-19 [1] RSPM (R 4.0.2)
yaml 2.2.1 2020-02-01 [1] RSPM (R 4.0.2)
`
This is an example of a reference that it cannot find:
@article{cheung2002, title = {Evaluating {{Goodness}}-of-{{Fit Indexes}} for {{Testing Measurement Invariance}}}, author = {Cheung, Gordon W. and Rensvold, Roger B.}, year = {2002}, month = apr, volume = {9}, pages = {233--255}, issn = {1070-5511, 1532-8007}, doi = {10.1207/S15328007SEM0902_5}, file = {/Users/aliciaarandadiaz/Zotero/storage/M9PZZQ7Y/Cheung y Rensvold - 2002 - Evaluating Goodness-of-Fit Indexes for Testing Mea.pdf}, journal = {Structural Equation Modeling: A Multidisciplinary Journal}, language = {en}, number = {2} }
Thank so much!
crsh
commented
3 years ago Hmm, sorry, I can't reproduce this problem. The following works for me. Could you try it out and let me know if you are still seeing the error?
---
title : "The title"
author:
- name : "First Author"
affiliation : "1"
corresponding : yes # Define only one corresponding author
address : "Postal address"
email : "my@email.com"
bibliography : ["references.bib"]
output : papaja::apa6_pdf
---
@cheung2002
\newpage
# ReferencesWith the following in references.bib:
@article{cheung2002, title = {Evaluating {{Goodness}}-of-{{Fit Indexes}} for {{Testing Measurement Invariance}}}, author = {Cheung, Gordon W. and Rensvold, Roger B.}, year = {2002}, month = apr, volume = {9}, pages = {233--255}, issn = {1070-5511, 1532-8007}, doi = {10.1207/S15328007SEM0902_5}, file = {/Users/aliciaarandadiaz/Zotero/storage/M9PZZQ7Y/Cheung y Rensvold - 2002 - Evaluating Goodness-of-Fit Indexes for Testing Mea.pdf}, journal = {Structural Equation Modeling: A Multidisciplinary Journal}, language = {en}, number = {2} }Thanks so much! I have tried and it works now. I have narrow down the error to a couple of equations I had in the doc - seems like the references were not the mail error. I didn't realize because the only warnings I saw were the citations... I must have written the equation with another version of R or something and that is why it does not run anymore. Thank you anyways!
Describe the bug When I try to knit a document that worked perfectly fine before, I now get the following error message:
Other files render fine, I am using the latest Pandoc version pandoc 2.11
This seems to affect only my
papajafiles. In the past I was able to render using RStudio 1.3x or 1.2x. Now this doesn't work either. Exact same error message. So it seems to me it's aPandocissue that is independent of RStudio but is connected topapaja.Is this correct? If so, can this be fixed?
Thanks for looking into this!