knitr: process ANSI color codes

[GegznaV]

Bug description

Colors or R warnings (see 1 in the figure) in Quarto are marked with symbols (see 2, 3). These symbols must not be present.

Steps to reproduce

```{r}
with(airquality, table(OzHi = Ozone > 80, Month)) |> 
  DescTools::Desc(plotit = FALSE)

### Expected behavior

No extra symbols of color codes.  e.g.

/.../ Mantel-Haenszel Chi-squared: X-squared = 1.1306, df = 1, p-value = 0.2876

Warning message: Exp. counts < 5: Chi-squared approx. may be incorrect!!

Contingency Coeff. 0.287 Cramer's V 0.300 Kendall Tau-b 0.065

/.../

If possible, the colored output would be appreciated too. But at least no extra symbols would be nice too.

### Actual behavior

/.../ Mantel-Haenszel Chi-squared: X-squared = 1.1306, df = 1, p-value = 0.2876  Warning message: Exp. counts < 5: Chi-squared approx. may be incorrect!!

 Contingency Coeff. 0.287 Cramer's V 0.300 Kendall Tau-b 0.065 /.../

![image](https://github.com/quarto-dev/quarto-cli/assets/12725868/d14fe68c-64e0-4b01-8211-bbecd15c5085)

### Your environment

    RStudio Edition : Desktop [Open Source]
    RStudio Version : 2023.6.0.401
    OS Version      : Windows 10 x64 (build 19045)
    R Version       : R version 4.3.0 (2023-04-21 ucrt)
    Quarto:         : 1.4.111

### Quarto check output

```bash
$ quarto check

[>] Checking versions of quarto binary dependencies...
      Pandoc version 3.1.2: OK
      Dart Sass version 1.55.0: OK
      Deno version 1.33.2: OK
[>] Checking versions of quarto dependencies......OK
[>] Checking Quarto installation......OK
      Version: 1.4.111
      Path: C:\Users\user\AppData\Local\Programs\Quarto\bin
      CodePage: 1257

[>] Checking basic markdown render....OK

[>] Checking Python 3 installation....OK
      Version: 3.9.15 (Conda)
      Path: C:/Users/user/anaconda3/python.exe
      Jupyter: 5.2.0
      Kernels: ir, julia-1.9, proj-ml-1, python3

(\) Checking Jupyter engine render....2023-05-30 19:08:32,583 - traitlets - ERROR - Failed to run command:
['C:\\Users\\user\\anaconda3\\envs\\proj-ml-1\\python.exe', '-m', 'ipykernel_launcher', '-f', 'C:\\Users\\user\\AppData\\Local\\Temp\\tmp5bcejmxp.json']
    PATH='C;C:\\Program Files\\Git\\Users\\user\\anaconda3;C;C:\\Program Files\\Git\\Users\\user\\anaconda3\\Scripts;C:\\Users\\user\\anaconda3;C:\\Users\\user\\anaconda3\\Library\\mingw-w64\\bin;C:\\Users\\user\\anaconda3\\Library\\usr\\bin;C:\\Users\\user\\anaconda3\\Library\\bin;C:\\Users\\user\\anaconda3\\Scripts;C:\\Users\\user\\anaconda3\\bin;C:\\Users\\user\\anaconda3\\condabin;C:\\Program Files\\RStudio\\resources\\app\\resources\\terminal\\bash\\bin;C:\\Program Files\\Git\\mingw64\\bin;C:\\Program Files\\Git\\usr\\local\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Program Files\\Git\\mingw64\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Program
Files\\RStudio\\resources\\app\\resources\\terminal\\bash\\bin;C:\\Users\\user\\anaconda3\\Scripts;C:\\Users\\user\\anaconda3;C:\\RBuildTools\\4.3\\x86_64-w64-mingw32.static.posix\\bin;C:\\RBuildTools\\4.3\\usr\\bin;C:\\Program Files\\R\\R-4.3.0\\bin\\x64;C:\\Program Files\\Microsoft MPI\\Bin;C:\\Python39\\Scripts;C:\\Python39;C:\\Rtools\\bin;C:\\ProgramData\\Boxstarter;C:\\Program Files\\ImageMagick-7.0.8-Q16;C:\\Rtools\\bin;C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath;C:\\Rtools\\mingw_32\\bin;C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v8.0\\bin;C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v8.0\\libnvvp;C:\\Program Files\\ImageMagick-6.9.3-Q16;C:\\ProgramData\\Oracle\\Java\\javapath;C:\\Program Files (x86)\\Intel\\iCLS Client;C:\\Program Files\\Intel\\iCLS Client;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0;C:\\Program Files\\Intel\\Intel(R) Management Engine Components\\DAL;C:\\Program Files (x86)\\Intel\\Intel(R) Management Engine Components\\DAL;C:\\Program Files\\Intel\\Intel(R) Management Engine Components\\IPT;C:\\Program Files (x86)\\Intel\\Intel(R) Management Engine Components\\IPT;C:\\Program Files\\Intel\\WiFi\\bin;C:\\Program Files\\Common Files\\Intel\\WirelessCommon;C:\\Program Files (x86)\\NVIDIA Corporation\\PhysX\\Common;C:\\Program Files (x86)\\Aspell\\bin;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0;C:\\Program Files (x86)\\QuickTime\\QTSystem;C:\\Program Files\\MiKTeX 2.9\\miktex\\bin\\x64;C:\\Program Files\\R\\R-3.4.0\\library\\tcltk\\libs\\x64;C:\\WINDOWS\\System32\\OpenSSH;C:\\Users\\user\\AppData\\Local\\Programs\\orca;C:\\Program
Files\\Microsoft VS Code\\bin;C:\\Program Files\\R\\R-3.5.1\\bin;C:\\ProgramData\\chocolatey\\bin;C:\\Program Files\\NVIDIA Corporation\\NVIDIA NvDLISR;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0;C:\\WINDOWS\\System32\\OpenSSH;C:\\Program Files\\SourceGear\\Common\\DiffMerge;C:\\Program Files\\Git LFS;C:\\Program Files\\TortoiseSVN\\bin;C:\\Program Files\\dotnet;C:\\Program Files (x86)\\Common Files\\Acronis\\VirtualFile;C:\\Program Files (x86)\\Common Files\\Acronis\\VirtualFile64;C:\\Program Files (x86)\\Common Files\\Acronis\\SnapAPI;C:\\Program Files\\Crucial\\Crucial Storage Executive;C:\\Program Files (x86)\\Microsoft SQL Server\\150\\Tools\\Binn;C:\\Program Files\\Microsoft SQL Server\\150\\Tools\\Binn;C:\\Program Files (x86)\\Microsoft SQL Server\\150\\DTS\\Binn;C:\\Program Files\\Microsoft SQL Server\\150\\DTS\\Binn;C:\\Program Files\\Microsoft SQL Server\\Client SDK\\ODBC\\170\\Tools\\Binn;C:\\Program Files\\Azure Data Studio\\bin;C:\\Program Files\\PuTTY;C:\\Program Files\\gs\\gs9.56.1\\bin;C:\\Program Files\\Docker\\Docker\\resources\\bin;C:\\ProgramData\\DockerDesktop\\version-bin;C:\\Program Files\\gnuplot\\bin;C:\\Program Files (x86)\\Meld;C:\\Program Files\\nodejs;C:\\Program Files\\Git\\cmd;C:\\Program Files\\PowerShell\\7;C:\\Users\\user\\AppData\\Local\\P[>] Checking Jupyter engine render....OK

[mcanouil]

The output is not really handle by Quarto (you'll get the same thing with rmarkdown). If you want special formatting, you'll have to use chunk hooks for example. Related discussion for cli: #3957.

Here is a partial example to get you started:

---
title: "Quarto Playground"
format: html
---

This is a playground for Quarto.

```{r}
library(knitr)
knit_print.Desc <- function(x, options) {
  fansi::sgr_to_html(
    x = paste(capture.output(DescTools:::print.Desc(x)), collapse = "\n"),
    warn = FALSE,
    term.cap = "256"
  )
}
registerS3method("knit_print", "Desc", knit_print.Desc)

#| output: asis
library(DescTools)
DescTools::Desc(with(airquality, table(OzHi = Ozone > 80, Month)), plotit = FALSE)

[mcanouil]

Closing this, as it is not planned to handle any possible output, especially because, the same would need to be done for all R packages and the same for any supported languages.

[cscheid]

I'm going to reopen this as an enhancement issue: we already process ANSI color code in jupyter output, and it might be possible to do something in the knitr side as well.

[mcanouil]

Oh really, there have been few discussion about knitr and ansi colour, especially for cli. Great news!

@cscheid Thanks for correcting me on this 😉

[cscheid]

See https://github.com/quarto-dev/quarto-cli/issues/4204. Even in Julia our support isn't great, and ideally we'd fix it across all engines.

[GegznaV]

@mcanouil, thank you for your insights. Still, iId like to clarify 2 things:

1) In:

```{r}
#| output: asis
library(DescTools)
DescTools::Desc(with(airquality, table(OzHi = Ozone > 80, Month)), plotit = FALSE)

Did you mean #| results: asis?

2) Using the hook, is this a reasonable way to remove ANSI codes:

```{r}
library(knitr)
knit_print.Desc <- function(x, options) {
  stringr::str_remove_all(
    paste(capture.output(DescTools:::print.Desc(x)), collapse = "\n"),
    "\\x1b\\[[0-9;]*m"

  )
}
registerS3method("knit_print", "Desc", knit_print.Desc)

[mcanouil]

1. Same thing output is Quarto, but both are supported internally by Quarto.
2. Whatever works for you, note that the code (even mine) messes up the "table" in the output.

From the code, it seems, one could use options("crayon.enabled" = TRUE/FALSE), but it does not seem to actually work for all ANSI colouring (definitely a bug in DescTools).

FALSE	TRUE

[cderv]

we already process ANSI color code in jupyter output, and it might be possible to do something in the knitr side as well.

@cscheid usually the R packages that does the ANSI color (like cli) usually deactivate them when knitting is detected. So there is nothing to fix in Quarto I believe. But if we want to bring the feature in Quarto to be compatible with knitr, we could make the other tools reactivate it when in Quarto. (or happy to bring the feature in R Markdown too if possible).

Interesting to work on this with you when we do.

If possible, the colored output would be appreciated too. But at least no extra symbols would be nice too.

This is really an upstream issue here: Desctool package is the one outputing this characters using cat() https://github.com/AndriSignorell/DescTools/blob/214a6615769fb4c2d39734c88595d968912ce72b/R/Desc.R#L1388-L1390

    if (!x$approx.ok) {
      cat("\033[31m\nWarning message:\n  Exp. counts < 5: Chi-squared approx. may be incorrect!!\n\033[39m")
    }

They are not using cli or other tools that knows when deactivate ANSI escape characters that are meant for Terminal and not for document like HTML and PDF.

So an issue should be open in DescTool IMO as they are outputing character you don't want in your output document. Even if we do support it in Quarto, it will be an issue for other documentation system. @GegznaV are you willing to open one there ?

Quarto or R Markdown are just showing what the package is cat()-ing.

[cderv]

This is really an upstream issue here: Desctool package is the one outputing this characters using cat() AndriSignorell/DescTools@214a661/R/Desc.R#L1388-L1390

FWIW, this specific issue have been solved in DescTools directly. See https://github.com/AndriSignorell/DescTools/commit/822c734abb3789c95c0cee99d5aea82f62389016#

[olivroy]

pandoc 3.4 was just released and it includes a new ANSI output format. Wonder if this could be levraged by Quarto for this (when Quarto uses pandoc 3.4).

New output format: ansi (for formatted console output) (Evan Silberman). Most Pandoc elements are supported and printed in a reasonable way, if not always ideally. This version does no detection of terminal capabilities, nor does it fall back to different output styles for less-capable terminals.

[cderv]

I saw that. My understanding is that it is an output format to format output for console. Same thing that what cli is doing in R, and what we get in knitr. What is required here is a ANSI Terminal Output parser (in a way) that would know how to convert for suitable HTML and other output.

So I don't think it could be useful as is.