org-indent-initialize-agent error when importing rmarkdown document

[xxmissingnoxx]

Thank you for sharing this project with the Emacs community.

I stumbled across an error "Error running timer 'org-indent-initialize-agent'" while importing an rmarkdown file into the org-mode format. A screenshot is attached below. Is this something you've encountered before? I'm currently using Doom emacs.

[tecosaur]

Off the bat, I have no idea, and I can't say I hope to reproduce this. Is this occuring with a particular file?

[tecosaur]

@xxmissingnoxx you're going to have to give me something to work with :P

[xxmissingnoxx]

I apologize for going dark there. I've been trying to create a minimal example which triggers the behavior. I've seen this behavior on 3 out of 3 files which are actual analyses so far. I tend to use the same format for my r notebook analyses that may not be too surprising if it has something to do with the content of the file.

I created a small example:

Testing

library(data.table)
library(ggplot2)
library(rstanarm) 

head(mtcars)

This yields:

• Testing =n{r import} library(data.table) library(ggplot2) library(rstanarm)=

=n{r head} head(mtcars)=

This does not trigger the error but similar behavior seems to appear in the more complicated files. The appearance of the letter "n" happens regularly. You should probably look at this post in the raw form as Github is rendering the rmd content. This is how it looks in the editor:

I used the command "org-pandoc-import-rmarkdown-as-org".

[tecosaur]

Thanks :smiley:. By the way, what version of Org are you running?

[xxmissingnoxx]

I realized I could try this without the named code blocks. I got different results. Assuming the code blocks have no name, should babel code blocks be the expected result or the example blocks generated below? I'm not sure if this is another clue or intended. It looks like Org 9.5 when I use the org-version command. I'm still getting acclimated to Emacs and Org stuff.

[xxmissingnoxx]

I've been tinkering with this off and on. It looks as though there are multiple factors in play.

The problem lies with your rmarkdown.lua filter. This problem occurs without any org or emacs in the mix. You can see this if you try to call pandoc on the command line with the filter. Rmarkdown is not compliant with any form of markdown supported by pandoc and thus is not appropriately handled by the AST and the lua filter. Rmarkdown is not a form of markdown that pandoc understands... but pandoc markdown is.

I think you can use an approximation of your existing code if you take a different approach.

1. First, use

   library(rmarkdown)
   render("test.rmd",md_document(variant="markdown"))

   to convert the rmarkdown to the pandoc flavor of markdown which is equipped to handle both backticks and fenced code attributes and classes as mentioned here,

2. Now that you have pandoc markdown, you can take advantage of the elem.attr and elem.classes attributes and get rid of (some) your regular expressions. Update your filter accordingly.
3. Apply your filter when converting from pandoc markdown to org.

I have a basic version of this working. The current problem with this approach is that the current settings mean losing chunk option information (ex. eval=FALSE). I suspect there's a way to get rmarkdown to dump that in the pandoc markdown too but experimentation, or perhaps posting an issue in their repo may be the next step.

[tecosaur]

Thanks for following up @xxmissingnoxx. I'm using the following test document, I hope like yours. However, I can't reproduce your issue.

I'm thinking perhaps it has something to do with the pre-processor? Do you have sed available?

[xxmissingnoxx]

I have sed available. I'm on a Mac so apparently you can't get the version number, but it works well enough to yell at me about it:

sed: illegal option -- -
usage: sed script [-Ealn] [-i extension] [file ...]
       sed [-Ealn] [-i extension] [-e script] ... [-f script_file] ... [file ...]

Pandoc version:

pandoc 2.11.3.2
Compiled with pandoc-types 1.22, texmath 0.12.1, skylighting 0.10.2,
citeproc 0.3.0.3, ipynb 0.1.0.1

Command line call:

pandoc --lua-filter=./org-pandoc-import/filters/rmarkdown.lua -f markdown -t org -o rmd.org test.md

Test file contents:

# Testing

Here is the first block:

```{r}
library(data.table)
library(ggplot2)
library(rstanarm) 

library(data.table)
library(ggplot2)
library(rstanarm) 

library(patchwork)
head(iris)

library(ggplot2)
library(rstanarm)
library(arm)
library(faraway) 

org-pandoc-import-rmarkdown-as-org output:

• Testing Here is the first block:

+BEGIN_EXAMPLE

library(data.table) library(ggplot2) library(rstanarm)

+END_EXAMPLE

=n{r .here} library(data.table) library(ggplot2) library(rstanarm)=

+BEGIN_EXAMPLE

library(patchwork) head(iris)

+END_EXAMPLE

=n{r eval=FALSE} library(ggplot2) library(rstanarm) library(arm) library(faraway)=

Command line call  output:

• Testing :PROPERTIES: :CUSTOM_ID: testing :END: Here is the first block:

+BEGIN_EXAMPLE

library(data.table) library(ggplot2) library(rstanarm)

+END_EXAMPLE

+BEGIN_EXAMPLE

library(patchwork) head(iris)

+END_EXAMPLE

+BEGIN_EXAMPLE

Sepal.Length Sepal.Width Petal.Length Petal.Width Species

1 5.1 3.5 1.4 0.2 setosa

2 4.9 3.0 1.4 0.2 setosa

3 4.7 3.2 1.3 0.2 setosa

4 4.6 3.1 1.5 0.2 setosa

5 5.0 3.6 1.4 0.2 setosa

6 5.4 3.9 1.7 0.4 setosa

+END_EXAMPLE

={r eval=FALSE} library(ggplot2) library(rstanarm) library(arm) library(faraway)=

+BEGIN_EXAMPLE

library(ggplot2) library(rstanarm) library(arm) library(faraway)

+END_EXAMPLE

+BEGIN_EXAMPLE

library(lme4)

+END_EXAMPLE

[tecosaur]

Thanks for the detailed example. I took your input file, ran the preprocessor on it, then ran pandoc with the filter. Here's what I found:

Left = input, Middle = preprocessed, Right = final

Could you try calling (call-process "sed" nil (list :file "preprocessedfile.Rmd") nil "s/^```{/```\\n{/" "infile.Rmd") and seeing what your output is? It should match the middle column above.

Thanks for your patience.

[xxmissingnoxx]

Thanks for the quick response. I get:

# Testing

Here is the first block:

```n{r}
library(data.table)
library(ggplot2)
library(rstanarm) 

library(data.table)
library(ggplot2)
library(rstanarm) 

library(patchwork)
head(iris)

library(ggplot2)
library(rstanarm)
library(arm)
library(faraway) 

Screenshots below if that helps.

<img width="875" alt="Screen Shot 2021-02-07 at 1 53 50 PM" src="https://user-images.githubusercontent.com/17442967/107156374-f0e5fc80-694b-11eb-80cd-8067c8f61be6.png">

<img width="761" alt="Screen Shot 2021-02-07 at 1 52 42 PM" src="https://user-images.githubusercontent.com/17442967/107156353-cf851080-694b-11eb-815f-02ef354b3bae.png">

[xxmissingnoxx]

Adding another slash before n seems to get your result. I'm not sure why you'd get different results though.

[tecosaur]

Interesting. I'm guessing this is a platform-specific difference. Let's see if I can't resolve this over the next few days :slightly_smiling_face:

[xxmissingnoxx]

That's what I'd guess too.

I feel like you shouldn't need to jump through the regex hoops to do this and there should be a way to do this with the rmarkdown package itself. I asked a question that probably won't be answered on SO to see if there's a nicer way to dump rmarkdown attributes. If that works, then it should be easier to simplify your code, at the cost of the R dependency, and maybe build on it (ex. use chunk label as named code block in org or preserve attributes if you want to export back out to rmarkdown).

Aside: What are you using to conceal the begin_src and end_src blocks. My emacs-fu is weak.

[tecosaur]

I'll return to sed later this week.

On your aside, I use doom and it's a result of a (ligatures +extra) module :slightly_smiling_face: