ChkTeX: warn of lines that are too long and therefore generate erroneous warnings

[tobiscode]

ChkTeX: warn of lines that are too long and therefore generate erroneous warnings

Preliminary questions

✔ Disable all the other extensions except for LaTeX Workshop, restart VS Code, and check that you can not see the requested feature. [Required] You still see this issue? Yes. ✔ Make sure to visit the wiki FAQ before requesting a feature. You visited the wiki? Yes. ✔ If your requested feature is with compiling a document (not having to do with finding the root file of a project), check first that you can compile manually. Not a compilation problem. ✔ Are you using VSCodium? No. ✔ Are you using the Snap or Flatpack versions of VS Code? No. ✔ Are you using LaTeX Workshop with VS Code Remote? No.

Is your requested feature related to a problem? Please describe.

When writing long sentences, especially when using lots of long commands, I've noticed that I sometimes got erroneous but serious-sounding warnings in the problem panel through ChkTeX, e.g. missing command brackets. Because compilation always worked fine, I got confused why ChkTeX would complain. I wasted a lot of time trying to figure it out, until I ran ChkTeX myself from the command line, which told me that ChkTeX has problems with long lines, and apparently just gives up linting. The LaTeX Workshop actually also relays this:

Linter for root file failed with exit code 2 and error:
chktex: WARNING -- ChkTeX does not handle lines over 509 bytes correctly.  Some errors and line numbers may be wrong in this file.

Now, since it's a long file, with many long lines, I had no idea where to look, so narrowing down the culprit line took some more time as well.

To reproduce

This minimal TeX file compiles and shows the problem:

\documentclass{article}
\begin{document}
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris tincidunt dapibus finibus. Sed in tempor magna. Aliquam venenatis ante vel elementum consectetur. Curabitur ligula nulla, efficitur vel venenatis vitae, placerat et purus. Etiam sed ligula id felis lobortis fringilla in a nibh. Donec ac volutpat lectus, eget ultricies diam. Sed nec turpis quis dui lobortis blandit vitae sit amet arcu. Maecenas felis nulla, facilisis eget dictum ac, pharetra a nibh. Nulla a pellentesque augue viverra fusce. \texttt{}
\end{document}

On the problem panel, I get:

14: Could not find argument for command. ChkTeX(14) [2, 18]

And running ChkTeX manually, I see that it ends parsing the line in the middle of the command:

chktex: WARNING -- ChkTeX does not handle lines over 509 bytes correctly.  Some errors and line num
bers may be wrong in this file.
Error 14 in test.tex line 2: Could not find argument for command.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris tincidunt dapibus finibus. Sed in t
empor magna. Aliquam venenatis ante vel elementum consectetur. Curabitur ligula nulla, efficitur ve
l venenatis vitae, placerat et purus. Etiam sed ligula id felis lobortis fringilla in a nibh. Donec
 ac volutpat lectus, eget ultricies diam. Sed nec turpis quis dui lobortis blandit vitae sit amet a
rcu. Maecenas felis nulla, facilisis eget dictum ac, pharetra a nibh. Nulla a pellentesque augue vi
verra fusce. \t

             ^^
One error printed; No warnings printed; No user suppressed warnings; No line suppressed warnings.  
See the manual for how to suppress some or all of these warnings/errors.

And when the parser ends between the curly brackets, the warning will be that there is a bracket missing.

Describe the solution you'd like

So of course obviously the best solution would be if ChkTeX would fix this upstream. It seems to be a known, yet terribly forgotten-about bug which was last talked about in 2012, and still not fixed in the last ChkTeX update from 2016. I haven't coded C in forever, it looks like a somewhat complex project, and somewhat stale, so I don't want to take time to relearn C, try to fix it, just to have it become a stale PR. But maybe by me posting this here someone who does know more about the ChkTeX code can see this and find an easy fix? I also found this related issue description.

In any case, what I did for myself, was to hack together a VS Code extension that just checks the currently open file for long lines, and writes the line numbers to a new output window, where I guess the relevant parts are:

var fpath = vscode.window.activeTextEditor?.document.uri.fsPath;

let vsout = vscode.window.createOutputChannel("chktextlinelength");

vsout.appendLine(`Checking file ${fpath}`)

let iline : number = 1;

var lineReader = require('readline').createInterface({
    input: require('fs').createReadStream(fpath, {encoding: 'utf-8'})
});

lineReader.on('line', function (line: any) {
    let linelen : number = (new TextEncoder().encode(line)).length;
    if (linelen > 509) {
        vsout.appendLine(`Line ${iline} is too long (${linelen} bytes):`)
        vsout.appendLine(`  ${String(line).substring(0, 60)}...\n`)
    }
    iline ++;
});

(Sorry for the bad TypeScript, this is literally the first time I tried this language.)

(Through my internet search, it actually turns out that the limit of 509 bytes in my case is actually not uniform, since it depends on the compilation parameters of ChkTeX, and I read that other users on the interwebs have it at 1024 bytes, so depending on that reproducers might need to add more characters.)

The code gives me:

Checking file c:\Data\test\test.tex
Line 3 is too long (517 bytes):
  Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mau...

This finally helped me find the lines in my source file that were too long.

It looks easy enough to integrate a similar option into the LaTeX Workshop, probably not scanning the file directly but using something like VS Code's getText function, and not creating a new output panel every time... And probably there's a much easier way to just add the warning or info into the problem panel instead an output console so that everything is neatly in one place.

A maybe even easier, although less informative thing one could do is to check for this particular message (see the log at the top) while parsing ChkTeX's error code and return message (which LaTeX Workshop does anyway), and then just put a generic warning into the problem panel that this is something the user should check if they see erroneous warnings. That avoids the document parsing, but still helps the user not go down the rabbit hole trying to find non-existent problems in their LaTeX file.

Logs

LaTeX Workshop Output

[11:53:51] Initializing LaTeX Workshop.
[11:53:51] Extension root: c:\Users\%USER%\.vscode\extensions\james-yu.latex-workshop-8.19.2
[11:53:51] $PATH: ...
[11:53:51] $SHELL: undefined
[11:53:51] latex-workshop.intellisense.update.aggressive.enabled: false
[11:53:51] latex-workshop.intellisense.update.delay: 1000
[11:53:51] latex-workshop.latex.autoBuild.run: "never"
[11:53:51] latex-workshop.latex.outDir: "%DIR%"
[11:53:51] latex-workshop.latex.recipes: [{"name":"latexmk 🔃","tools":["latexmk"]},{"name":"latexmk (latexmkrc)","tools":["latexmk_rconly"]},{"name":"latexmk (lualatex)","tools":["lualatexmk"]},{"name":"pdflatex ➞ bibtex ➞ pdflatex × 2","tools":["pdflatex","bibtex","pdflatex","pdflatex"]},{"name":"Compile Rnw files","tools":["rnw2tex","latexmk"]},{"name":"Compile Jnw files","tools":["jnw2tex","latexmk"]},{"name":"tectonic","tools":["tectonic"]}]
[11:53:51] latex-workshop.latex.tools: [{"name":"latexmk","command":"latexmk","args":["-synctex=1","-interaction=nonstopmode","-file-line-error","-pdf","-outdir=%OUTDIR%","%DOC%"],"env":{}},{"name":"lualatexmk","command":"latexmk","args":["-synctex=1","-interaction=nonstopmode","-file-line-error","-lualatex","-outdir=%OUTDIR%","%DOC%"],"env":{}},{"name":"latexmk_rconly","command":"latexmk","args":["%DOC%"],"env":{}},{"name":"pdflatex","command":"pdflatex","args":["-synctex=1","-interaction=nonstopmode","-file-line-error","%DOC%"],"env":{}},{"name":"bibtex","command":"bibtex","args":["%DOCFILE%"],"env":{}},{"name":"rnw2tex","command":"Rscript","args":["-e","knitr::opts_knit$set(concordance = TRUE); knitr::knit('%DOCFILE_EXT%')"],"env":{}},{"name":"jnw2tex","command":"julia","args":["-e","using Weave; weave(\"%DOC_EXT%\", doctype=\"tex\")"],"env":{}},{"name":"jnw2texmintex","command":"julia","args":["-e","using Weave; weave(\"%DOC_EXT%\", doctype=\"texminted\")"],"env":{}},{"name":"tectonic","command":"tectonic","args":["--synctex","--keep-logs","%DOC%.tex"],"env":{}}]
[11:53:51] latex-workshop.viewer.pdf.internal.keyboardEvent: "auto"
[11:53:51] Creating a new file watcher.
[11:53:51] watcherOptions: {"useFsEvents":false,"usePolling":false,"interval":300,"binaryInterval":1000,"awaitWriteFinish":{"stabilityThreshold":250}}
[11:53:51] Creating PDF file watcher.
[11:53:51] watcherOptions: {"useFsEvents":false,"usePolling":false,"interval":300,"binaryInterval":1000,"awaitWriteFinish":{}}
[11:53:51] Creating Bib file watcher.
[11:53:51] watcherOptions: {"useFsEvents":false,"usePolling":false,"interval":300,"binaryInterval":1000,"awaitWriteFinish":{"stabilityThreshold":250}}
[11:53:51] pdflatex is provided by MiKTeX
[11:53:51] [Server] Creating LaTeX Workshop http and websocket server.
[11:53:51] LaTeX Workshop initialized.
[11:53:51] Trigger characters for intellisense of LaTeX documents: ["\\","{",",","(","["]
[11:53:51] Bibtex format config: {"tab":"  ","case":"lowercase","left":"{","right":"}","trailingComma":false,"sort":["key"],"alignOnEqual":true,"sortFields":false,"fieldsOrder":[]}
[11:53:51] Current workspace folders: ["file:///c:/Data/test"]
[11:53:51] Current workspaceRootDir: file:///c:/Data/test
[11:53:51] Found root file from active editor: c:\Data\test\test.tex
[11:53:51] Root file changed: from undefined to c:\Data\test\test.tex
[11:53:51] Start to find all dependencies.
[11:53:51] Root file languageId: latex
[11:53:51] [Server] Server successfully started: {"address":"127.0.0.1","family":"IPv4","port":61430}
[11:53:51] Reset file watcher.
[11:53:51] Parsing a file and its subfiles: c:\Data\test\test.tex
[11:53:51] Parse fls file.
[11:53:51] Cannot find fls file: c:\Data\test\test.fls
[11:53:51] Linter for root file started.
[11:53:51] Linter for root file running command chktex with arguments ["-wall","-n22","-n30","-e16","-q","-n37","-f%f:%l:%c:%d:%k:%n:%m\n","c:\\Data\\test\\test.tex"]
[11:53:51] Added to file watcher: c:\Data\test\test.tex
[11:53:51] Snippet data loaded.
[11:53:52] Checking for duplicate labels: c:\Data\test\test.tex.
[11:53:52] Linter for root file failed with exit code 2 and error:
chktex: WARNING -- ChkTeX does not handle lines over 509 bytes correctly.  Some errors and line numbers may be wrong in this file.

[11:53:52] The .chktexrc file not found.
[11:53:52] Linter log parsed with 1 messages.
[11:53:53] Manager.fileWatcher.getWatched: {"c:\\Data\\test":["test.tex"]}
[11:53:53] Manager.filesWatched: ["c:\\Data\\test\\test.tex"]
[11:54:04] Current workspace folders: ["file:///c:/Data/test"]
[11:54:04] Current workspaceRootDir: file:///c:/Data/test
[11:54:04] Found root file from active editor: c:\Data\test\test.tex
[11:54:04] Keep using the same root file: c:\Data\test\test.tex
[11:54:04] Linter for root file started.
[11:54:04] Linter for root file running command chktex with arguments ["-wall","-n22","-n30","-e16","-q","-n37","-f%f:%l:%c:%d:%k:%n:%m\n","c:\\Data\\test\\test.tex"]
[11:54:05] Linter for root file failed with exit code 2 and error:
chktex: WARNING -- ChkTeX does not handle lines over 509 bytes correctly.  Some errors and line numbers may be wrong in this file.

[11:54:05] The .chktexrc file not found.
[11:54:05] Linter log parsed with 1 messages.
[11:54:06] Manager.fileWatcher.getWatched: {"c:\\Data\\test":["test.tex"]}
[11:54:06] Manager.filesWatched: ["c:\\Data\\test\\test.tex"]
[11:54:08] LOG command invoked: default
[11:54:08] Current workspace folders: ["file:///c:/Data/test"]
[11:54:08] Current workspaceRootDir: file:///c:/Data/test
[11:54:08] Found root file from active editor: c:\Data\test\test.tex
[11:54:08] Keep using the same root file: c:\Data\test\test.tex
[11:54:08] Linter for root file started.
[11:54:08] Linter for root file running command chktex with arguments ["-wall","-n22","-n30","-e16","-q","-n37","-f%f:%l:%c:%d:%k:%n:%m\n","c:\\Data\\test\\test.tex"]
[11:54:08] Linter for root file failed with exit code 2 and error:
chktex: WARNING -- ChkTeX does not handle lines over 509 bytes correctly.  Some errors and line numbers may be wrong in this file.

[11:54:08] The .chktexrc file not found.
[11:54:08] Linter log parsed with 1 messages.
[11:54:10] Manager.fileWatcher.getWatched: {"c:\\Data\\test":["test.tex"]}
[11:54:10] Manager.filesWatched: ["c:\\Data\\test\\test.tex"]

Developer Tools Console

This is probably unrelated but still (only message in dev console):

console.ts:137 [Extension Host] (node:3092) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.(Use `Code --trace-deprecation ...` to show where the warning was created)

Desktop

• OS: Windows 10
• VS Code version: 1.58.0 (user setup)
• Extension version: v8.19.2
• TeX distribution version: MiKTeX-pdfTeX 4.6.1 (MiKTeX 21.6)

[jlelong]

It is weird, I cannot reproduce the issue even with a line longer than 3000 characters. The line number is correct but the column number is wrong. I use chktex-1.7.6 from TexLive 2021.

I am afraid it is out of the scope of this extension to implement tricks to bypass a chktex bug. Note that you can always deactivate automatic linting "latex-workshop.chktex.enabled": false.

[tobiscode]

Interesting - I also have ChkTeX v1.7.6. So maybe there is a difference between TexLive and MikTeX?..

I do understand though in this case that this is out of scope (even if I think just deactivating ChkTeX in the extension settings isn't an actual workaround).

Anyway, I hope me posting this here at least helps others facing the same problem, and maybe someone with more C or TS coding experience will get just as annoyed at this as me to have a closer look 😉

Edit: just installed TeX Live 2021/W32TeX and the problem is the same. So maybe it's a Windows vs. Mac/Unix thing then?... This is what both give for xhktex --version:

ChkTeX v1.7.6 - Copyright 1995-96 Jens T. Berger Thielemann.
Compiled with POSIX extended regex support.

Edit 2: Yep, can confirm this problem doesn't happen on either a dedicated Linux or even WSL Linux...