search

James-Yu / LaTeX-Workshop

Boost LaTeX typesetting efficiency with preview, compile, autocomplete, colorize, and more.
MIT License
10.58k stars 527 forks source link

Trouble with LaTeX parsing related to listings and math mode #3922

Closed sorawee closed 1 year ago

sorawee commented 1 year ago

Please fill the following fields with a star (*) and provide as much related information as possible.

Pre-checks*

Please change the following [ ] to [x] for confirmation.

  • [x] The issue has not been reported in this repository.
  • [x] The issue remains after disabling all other extensions and restarting Visual Studio Code.
  • [x] The FAQ cannot address the issue.
  • [x] The issue is not related to compiling a document, or the document can be successfully compiled in the OS terminal but not in Visual Studio Code with this extension.

Environment*

Please write exact version numbers instead of descriptors such as latest.

  • Mac OS (M1)
  • Visual Studio Code Version: 1.78.2
  • LaTeX Workshop Version: 9.11.4, but it's actually commit c46296cd2f7258a (latest commit, installed from git source) with minor modification to debug the problem (see the section "Anything Else?" below)
  • TeX Distribution Version: TeX Live 2022

The Issue*

Please briefly describe the issue you come across.

  • When a listing contains a symbol $ (as a part of the language), the parser gets confused when trying to parse it. The whole extension appears to be unresponsive for 2^n unit of time when there are n parse errors in the document. During this unresponsiveness, entering a newline is totally lagged (see also #2637, #3853, etc) and saving appears to be lagged as well (though not as bad as entering). All functionalities (build, preview) are unusable.

Reproduction Steps

Please list out the steps to reproduce your bug. Include relevant environmental variables or any other configuration.

Consider the following file:

\documentclass{article}

\usepackage{listings}

\lstdefinelanguage{test}{
  morekeywords={<$>},
  alsoletter={<>$},
}

\begin{document}

\begin{center}
    \lstinline[language=test]{a <$> b}
\end{center}

\begin{center}
    \lstinline[language=test]{a <$> b}
\end{center}

\begin{center}
    \lstinline[language=test]{a <$> b}
\end{center}

\begin{center}
    \lstinline[language=test]{a <$> b}
\end{center}

\begin{center}
    \lstinline[language=test]{a <$> b}
\end{center}

\begin{center}
    \lstinline[language=test]{a <$> b}
\end{center}

\begin{center}
    \lstinline[language=test]{a <$> b}
\end{center}

\begin{center}
    \lstinline[language=test]{a <$> b}
\end{center}

\begin{center}
    \lstinline[language=test]{a <$> b}
\end{center}

\begin{center}
    \lstinline[language=test]{a <$> b}
\end{center}

\begin{center}
    \lstinline[language=test]{a <$> b}
\end{center}

\begin{center}
    \lstinline[language=test]{a <$> b}
\end{center}

\begin{center}
    \lstinline[language=test]{a <$> b}
\end{center}

\begin{center}
    \lstinline[language=test]{a <$> b}
\end{center}

\end{document}
  1. Make a random edit (adding "abc" before \end{document}).
  2. Save
  3. Observe that the extension is stuck for several seconds (~5-10s on my computer). I repeat \begin{center}...\end{center} high enough to make it evidently stuck on my computer, but not too high that it appears to hang forever. Adding more \begin{center}...\end{center} will make it hang longer (exponentially for each adding). Removing \begin{center}...\end{center} will make it hang less.
    • By "stuck", I meant what I mentioned above: entering and saving is lagged. Functionalities like building the project are not functional. Note that shift-enter for entering a newline still can be used.

Expected Behavior

What were you expecting to see? Include any relevant examples or documentation links.

  • The extension continues to work, and even if it does not, it should not disrupt VSCode.

Logs

LaTeX Workshop Output*

Please paste the whole log messages below, not parts of ones. The log should start with New log placeholder %WS1% registered.


[09:06:25.486][Logger] New log placeholder %WS1% registered for /Users/sorawee/projects/test-doc .
[09:06:25.486][Config] latex-workshop.latex.recipes: [{"name":"latexmk 🔃","tools":["latexmk"]},{"name":"latexmk (latexmkrc)","tools":["latexmk_rconly"]},{"name":"latexmk (lualatex)","tools":["lualatexmk"]},{"name":"latexmk (xelatex)","tools":["xelatexmk"]},{"name":"pdflatex ➞ bibtex ➞ pdflatex × 2","tools":["pdflatex","bibtex","pdflatex","pdflatex"]},{"name":"Compile Rnw files","tools":["rnw2tex","latexmk"]},{"name":"Compile Jnw files","tools":["jnw2tex","latexmk"]},{"name":"tectonic","tools":["tectonic"]}] .
[09:06:25.935][Config] latex-workshop.latex.recipe.default: "lastUsed" .
[09:06:25.936][Config] latex-workshop.latex.tools: [{"name":"latexmk","command":"latexmk","args":["-synctex=1","-interaction=nonstopmode","-file-line-error","-pdf","-outdir=%OUTDIR%","%DOC%"],"env":{}},{"name":"lualatexmk","command":"latexmk","args":["-synctex=1","-interaction=nonstopmode","-file-line-error","-lualatex","-outdir=%OUTDIR%","%DOC%"],"env":{}},{"name":"xelatexmk","command":"latexmk","args":["-synctex=1","-interaction=nonstopmode","-file-line-error","-xelatex","-outdir=%OUTDIR%","%DOC%"],"env":{}},{"name":"latexmk_rconly","command":"latexmk","args":["%DOC%"],"env":{}},{"name":"pdflatex","command":"pdflatex","args":["-synctex=1","-interaction=nonstopmode","-file-line-error","%DOC%"],"env":{}},{"name":"bibtex","command":"bibtex","args":["%DOCFILE%"],"env":{}},{"name":"rnw2tex","command":"Rscript","args":["-e","knitr::opts_knit$set(concordance = TRUE); knitr::knit('%DOCFILE_EXT%')"],"env":{}},{"name":"jnw2tex","command":"julia","args":["-e","using Weave; weave(\"%DOC_EXT%\", doctype=\"tex\")"],"env":{}},{"name":"jnw2texmintex","command":"julia","args":["-e","using Weave; weave(\"%DOC_EXT%\", doctype=\"texminted\")"],"env":{}},{"name":"tectonic","command":"tectonic","args":["--synctex","--keep-logs","%DOC%.tex"],"env":{}}] .
[09:06:26.82][Config] latex-workshop.latex.autoBuild.run: "onSave" .
[09:06:26.360][Manager] Set $LATEXWORKSHOP_DOCKER_LATEX: ""
[09:06:26.426][Server] Creating LaTeX Workshop http and websocket server.
[09:06:26.541][Server] Server successfully started: {"address":"127.0.0.1","family":"IPv4","port":64528} .
[09:06:26.565][Extension] Initializing LaTeX Workshop.
[09:06:26.566][Extension] Extension root: /Users/sorawee/.vscode/extensions/james-yu.latex-workshop-9.11.4
[09:06:26.566][Extension] $PATH: /Users/sorawee/.opam/default/bin:/Users/sorawee/projects/solvers:/Users/sorawee/.elan/bin:/opt/homebrew/bin:/opt/homebrew/opt:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/Users/sorawee/projects/racket/racket/bin:/usr/local/share/dotnet:~/.dotnet/tools
[09:06:26.566][Extension] $SHELL: /bin/zsh
[09:06:26.566][Extension] $LANG: en_US.UTF-8
[09:06:26.567][Extension] $LC_ALL: undefined
[09:06:26.567][Extension] process.platform: darwin
[09:06:26.567][Extension] process.arch: x64
[09:06:26.567][Extension] vscode.env.appName: Visual Studio Code
[09:06:26.567][Extension] vscode.env.remoteName: undefined
[09:06:26.567][Extension] vscode.env.uiKind: 1
[09:06:26.568][Extension] LaTeX Workshop initialized.
[09:06:26.575][Format][Bib] Bibtex format config: {"tab":"  ","case":"lowercase","left":"{","right":"}","trailingComma":false,"sort":["key"],"alignOnEqual":true,"sortFields":false,"fieldsOrder":[],"firstEntries":["string","xdata"]}
[09:06:26.578][Extension] Trigger characters for intellisense of LaTeX documents: ["\\",",","{"]
[09:06:26.594][Manager] Current workspace folders: ["file://%WS1%"]
[09:06:26.606][Manager] Found root file from active editor: %WS1%/doc.tex
[09:06:26.607][Manager] Root file changed: from undefined to %WS1%/doc.tex
[09:06:26.607][Manager] Start to find all dependencies.
[09:06:26.607][Manager] Root file languageId: latex
[09:06:26.607][Event] ROOT_FILE_CHANGED: "%WS1%/doc.tex"
[09:06:26.608][Cacher][Watcher] Reset.
[09:06:26.629][Cacher] Adding %WS1%/doc.tex .
[09:06:26.632][Cacher][Watcher] Watched %WS1%/doc.tex with a new watcher on %WS1% .
[09:06:26.633][Event] FILE_WATCHED: "%WS1%/doc.tex"
[09:06:26.654][Cacher] Caching %WS1%/doc.tex .
[09:06:26.656][Cacher] Updated inputs of %WS1%/doc.tex .
[09:06:26.656][Cacher] Parse LaTeX AST with fast-parse: %WS1%/doc.tex .
[09:06:26.660][Event] ROOT_FILE_SEARCHED
[09:06:26.680][Server] valdOrigin is http://127.0.0.1:64528
[09:06:26.692][Viewer][Panel] Restoring at column 1 with state {"pdfFileUri":"file://%WS1%/doc.pdf","scale":"auto","scrollMode":0,"sidebarView":0,"spreadMode":0,"scrollTop":0,"scrollLeft":0,"trim":0,"synctexEnabled":true,"autoReloadEnabled":true}.
[09:06:26.749][Viewer][Panel] Internal PDF viewer at http://127.0.0.1:64528/viewer.html?file=pdf..ZmlsZSUzQSUyRiUyRiUyRlVzZXJzJTJGc29yYXdlZSUyRnByb2plY3RzJTJGdGVzdC1kb2MlMkZkb2MucGRm .
[09:06:26.757][Cacher][Watcher] Watched %WS1%/doc.pdf with a new watcher on %WS1% .
[09:06:26.761][Event] FILE_WATCHED: "%WS1%/doc.pdf"
[09:06:27.123][Parser] Error in parsing LaTeX AST: \documentclass{article}

\usepackage{listings}

\lstdefinelanguage{test}{ morekeywords={<$>}, alsoletter={<>$}, }

\begin{document}

\begin{center} \lstinline[language=test]{a <$> b} \end{center}

\begin{center} \lstinline[language=test]{a <$> b} \end{center}

\begin{center} \lstinline[language=test]{a <$> b} \end{center}

\begin{center} \lstinline[language=test]{a <$> b} \end{center}

\begin{center} \lstinline[language=test]{a <$> b} \end{center}

\begin{center} \lstinline[language=test]{a <$> b} \end{center}

\begin{center} \lstinline[language=test]{a <$> b} \end{center}

\begin{center} \lstinline[language=test]{a <$> b} \end{center}

\begin{center} \lstinline[language=test]{a <$> b} \end{center}

\begin{center} \lstinline[language=test]{a <$> b} \end{center}

\begin{center} \lstinline[language=test]{a <$> b} \end{center}

\begin{center} \lstinline[language=test]{a <$> b} \end{center}

\begin{center} \lstinline[language=test]{a <$> b} \end{center}

\begin{center} \lstinline[language=test]{a <$> b} \end{center}

\end{document}: Expected "Big", "Bigg", "\", "autoref", "begin", "big", "bigg", "eqref", "label", "left", "linebreak", "newline", "ref", or "text" but "e" found. Location context: {"start":{"offset":188,"line":14,"column":2},"end":{"offset":189,"line":14,"column":3}} . [09:06:27.124][Cacher] Parsed LaTeX AST with LU in 467.47 ms: %WS1%/doc.tex . [09:06:32.616][Cacher] Parsed LaTeX AST in 5490.88 ms: %WS1%/doc.tex . [09:06:32.777][Cacher] Failed parsing LaTeX AST. [09:06:32.790][Cacher] Use RegExp to update elements of %WS1%/doc.tex . [09:06:32.993][DupLabel] Checking for duplicate labels: %WS1%/doc.tex . [09:06:33.5][Cacher] Updated elements of %WS1%/doc.tex . [09:06:33.19][Cacher] Updated bibs of %WS1%/doc.tex . [09:06:33.35][Cacher] Cached %WS1%/doc.tex . [09:06:33.39][Event] FILE_PARSED: "%WS1%/doc.tex" [09:06:33.74][Cacher] Parsing .fls %WS1%/doc.fls . [09:06:33.127][Cacher] Found .aux %WS1%/doc.tex from .fls %WS1%/doc.fls , parsing. [09:06:33.134][Cacher] Parsed .aux %WS1%/doc.tex . [09:06:33.156][Cacher] Parsed .fls %WS1%/doc.fls . [09:06:33.177][Structure] Structure force updated with 0 root sections for %WS1%/doc.tex . [09:06:33.198][Event] STRUCTURE_UPDATED [09:06:35.556][Viewer] Handle data type: open [09:06:35.827][Server] Preview PDF file: file://%WS1%/doc.pdf [09:06:37.809][Viewer] Handle data type: loaded [09:06:37.809][Event] VIEWER_PAGE_LOADED [09:06:38.74][Event] VIEWER_STATUS_CHANGED: {"pdfFileUri":"file://%WS1%/doc.pdf","scale":"auto","scrollMode":0,"sidebarView":0,"spreadMode":0,"scrollTop":0,"scrollLeft":0,"trim":0,"synctexEnabled":true,"autoReloadEnabled":true} [09:06:38.74][Event] VIEWER_STATUS_CHANGED: {"pdfFileUri":"file://%WS1%/doc.pdf","scale":"auto","scrollMode":0,"sidebarView":0,"spreadMode":0,"scrollTop":0,"scrollLeft":0,"trim":0,"synctexEnabled":true,"autoReloadEnabled":true} [09:06:38.75][Event] VIEWER_STATUS_CHANGED: {"pdfFileUri":"file://%WS1%/doc.pdf","scale":"auto","scrollMode":0,"sidebarView":0,"spreadMode":0,"scrollTop":0,"scrollLeft":0,"trim":0,"synctexEnabled":true,"autoReloadEnabled":true} [09:06:38.75][Event] VIEWER_STATUS_CHANGED: {"pdfFileUri":"file://%WS1%/doc.pdf","scale":"auto","scrollMode":0,"sidebarView":0,"spreadMode":0,"scrollTop":0,"scrollLeft":0,"trim":0,"synctexEnabled":true,"autoReloadEnabled":true} [09:06:38.80][Event] VIEWER_STATUS_CHANGED: {"pdfFileUri":"file://%WS1%/doc.pdf","scale":"auto","scrollMode":0,"sidebarView":0,"spreadMode":0,"scrollTop":0,"scrollLeft":0,"trim":0,"synctexEnabled":true,"autoReloadEnabled":true}


### Developer Tools Console
> _Please paste the whole log messages below, not parts of ones. This console logs can sometimes be very important in many cases. To access the log, click `Help` -> `Toggle Developer Tools` -> `Console`._

This is the same information I will provide below, but the information below is more detailed, so I will skip this section.

## Anything Else?
From the "Window" output, here's the log:

package.json (LOSES): inlineCompletionsAdditions, terminalDataWriteEvent product.json (WINS): inlineCompletionsAdditions 2023-06-07 09:06:28.257 [info] [perf] Render performance baseline is 31ms 2023-06-07 09:06:30.562 [info] UNRESPONSIVE extension host: starting to profile NOW 2023-06-07 09:06:33.217 [info] UNRESPONSIVE extension host: received responsive event and cancelling profiling session 2023-06-07 09:06:33.747 [warning] UNRESPONSIVE extension host: 'james-yu.latex-workshop' took 91.6148157152655% of 2441.839ms, saved PROFILE here: 'file:///var/folders/8v/ssfgpb5d67n4jrvmszsh7y800000gn/T/exthost-032bf8.cpuprofile'


Note that if I try to save/enter during the unresponsiveness, these messages will be additionally logged:

2023-06-07 09:14:40.224 [error] Error: Aborted onWillSaveTextDocument-event after 1750ms at vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1844:13556 2023-06-07 09:14:41.264 [info] UNRESPONSIVE extension host: starting to profile NOW 2023-06-07 09:14:42.395 [error] Error: Aborted onWillSaveTextDocument-event after 1750ms at vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:1844:13556 2023-06-07 09:14:45.229 [info] UNRESPONSIVE extension host: received responsive event and cancelling profiling session 2023-06-07 09:14:46.029 [warning] UNRESPONSIVE extension host: 'james-yu.latex-workshop' took 96.41605360944111% of 3840.424ms, saved PROFILE here: 'file:///var/folders/8v/ssfgpb5d67n4jrvmszsh7y800000gn/T/exthost-2e0657.cpuprofile' 2023-06-07 09:14:48.786 [info] UNRESPONSIVE extension host: starting to profile NOW 2023-06-07 09:14:53.345 [info] UNRESPONSIVE extension host: received responsive event and cancelling profiling session 2023-06-07 09:14:54.172 [warning] UNRESPONSIVE extension host: 'james-yu.latex-workshop' took 93.68096632324936% of 4297.723ms, saved PROFILE here: 'file:///var/folders/8v/ssfgpb5d67n4jrvmszsh7y800000gn/T/exthost-96cb5a.cpuprofile' 2023-06-07 09:14:57.071 [info] UNRESPONSIVE extension host: starting to profile NOW 2023-06-07 09:15:02.421 [warning] UNRESPONSIVE extension host: 'james-yu.latex-workshop' took 98.00339270365677% of 4923.418ms, saved PROFILE here: 'file:///var/folders/8v/ssfgpb5d67n4jrvmszsh7y800000gn/T/exthost-d6572b.cpuprofile' 2023-06-07 09:15:08.406 [info] UNRESPONSIVE extension host: starting to profile NOW 2023-06-07 09:15:13.908 [warning] UNRESPONSIVE extension host: 'james-yu.latex-workshop' took 98.36969268833501% of 4944.77ms, saved PROFILE here: 'file:///var/folders/8v/ssfgpb5d67n4jrvmszsh7y800000gn/T/exthost-0ebf1b.cpuprofile'



I would really appreciate it if logging at https://github.com/James-Yu/LaTeX-Workshop/blob/c46296cd2f7258a15db1fc9e5ccbe5f0b4d93abd/src/components/parser.ts#L36 also contains information about what is being parsed (file name, a part of file content, etc). I was initially stumped for a really long time for what causes the error in my large project. It's only when I modified this line to include the string `s` that I understood which file causes the problem, leading me to create the minimal example in this issue report.
jlelong commented 1 year ago

Unfortunately, there is no easy of declaring the content of a macro as verbatim, which is precisely what would solve the current issue. It is possible to declare environments as verbatim using latex-workshop.latex.verbatimEnvs.

Achieving the same for macros is far more difficult as finding the closing brace delimiting the macro can be tricky because of brace nesting. We may give it a try and see how robust it is.

sorawee commented 1 year ago

Personally, I think it's fine that the file will fail to parse. What I should have made more clear is that I am more concerned with basic functionalities (building, viewing pdf, editing, saving), which should not be affected by this issue.

Can we impose a time limit on the parser call, and don't do advanced functionalities on the file if the limit exceeds?

jlelong commented 1 year ago

What is the value of latex-workshop.intellisense.fastparse.enabled? If it is false, can you try to set it to true?

sorawee commented 1 year ago

It is already true.

James-Yu commented 1 year ago

It’s indeed unified-latex that lags (5000+ms). I will look into the issue.

In principle, parsing should not prevent users from other actions. I’m yet to figure out what’s wrong with that.

James-Yu commented 1 year ago

We have a temporary and dirty (but works well) solution: For now, all $ signs (not \$) is replaced by a harmless | before parsing with unified-latex. I have created an issue to the upstream https://github.com/siefkenj/unified-latex/issues/35 and will keep an eye.

James-Yu commented 1 year ago

2e585b8 is not a viable solution. I will work at the upstream https://github.com/siefkenj/unified-latex/issues/35 to solve this issue once and for all.

James-Yu commented 1 year ago

Fixed in bb16132