sagemathinc / cocalc

CoCalc: Collaborative Calculation in the Cloud
https://CoCalc.com
Other
1.14k stars 207 forks source link

rmarkdown generator -- invalid document #7575

Open williamstein opened 1 month ago

williamstein commented 1 month ago
  1. Select the "Publication-Ready Manuscript" template and use "GPT-4 omni".

  2. Get a broken document because the first three backticks are missing. This seems pretty serious and likely like a bug in your parser, etc.:

{r}
set.seed(123)
data <- data.frame(
  x = rnorm(100),
  y = rnorm(100)
)
head(data)

Results

Present the key findings of the study.

Figure

library(ggplot2)
ggplot(data, aes(x = x, y = y)) +
  geom_point() +
  theme_minimal()


Note above how the document starts with {r} instead of three backticks, then {r}.

I tested multiple times and got this.

I also just tried with the markdown tutorial and got this where the document is missing the first few characters.  This is a pretty serious bug and certainly a release blocker.

<img width="772" alt="image" src="https://github.com/sagemathinc/cocalc/assets/1276278/f0e996ce-a744-4760-ac05-05d8d2e531e8">
williamstein commented 1 month ago

mostly fixed by #7577 7577

williamstein commented 1 month ago

I just tried again with GPT4o and selecting "Financial Report" and the document is pretty messed up. It produced this:


title: "Financial Report" output: html_document: toc: true fig_caption: true number_sections: true

Overview

...



i.e., now it has triple backticks at the beginning but shouldn't have them.  How annoying.  When I removed the leading ``` it then 100% works.

So I'll re-open this, but I'm not considering it a release blocker.