Error format documentation for UIs / more consistent error format

[cpitclaudel]

Hi there,

I co-maintain Flycheck, an Emacs UI for linters and compilers (Flycheck runs compilers in the background and displays errors as squiggly underlines).

I'm in the process of updating the checker definition for jsonnet and I'm having trouble figuring out a regular expression that covers all error messages returned by jsonnet (in general we prefer when tools can report their errors in a machine-friendly format, such as JSON or XML, but IIUC jsonnet only supports human-readable output; correct?)

Is there documentation somewhere (sorry if I missed it!) about the format that error messages are expected to conform to? I went through the .golden outputs in the test directory, but it seems that some errors do not include line or column numbers at all (e.g. https://github.com/google/jsonnet/blob/master/test_suite/error.function_no_default_arg.jsonnet.golden). I understand that static errors have a single location and that runtime errors have a stack trace; correct?

It would be great if jsonnet could consistently print at least one location for each error, but we'll adjust even if it can't :)

Thanks!

[seh]

My patch against jsonnet-mode (mgyucht/jsonnet-mode#12) includes such regular expressions for use with compile-mode. I refined those against the C++ Jsonnet port. Some of the Go port's error messages differ.

[cpitclaudel]

Nifty! Flycheck computes these automatically from its own error patterns, so I get this:

(("^STATIC ERROR: \\(?:\\(?1:.+?\\)\\):\\(?:\\(?:\\(?2:[[:digit:]]+\\)\\):\\(?:\\(?3:[[:digit:]]+\\)\\)\\(?:-\\(?:\\(?7:[[:digit:]]+\\)\\)\\)?\\|(\\(?:\\(?2:[[:digit:]]+\\)\\):\\(?:\\(?3:[[:digit:]]+\\)\\))-(\\(?:\\(?6:[[:digit:]]+\\)\\):\\(?:\\(?7:[[:digit:]]+\\)\\))\\): \\(?:\\(?4:.+\\)\\)$" 1 #1=(2 . 6)
  #2=(3 . 7)
  2)
 ("^RUNTIME ERROR: \\(?:\\(?4:.+\\)\\)
\\(?:   \\(?:\\(?1:.+?\\)\\):\\(?:\\(?:\\(?2:[[:digit:]]+\\)\\):\\(?:\\(?3:[[:digit:]]+\\)\\)\\(?:-\\(?:\\(?7:[[:digit:]]+\\)\\)\\)?\\|(\\(?:\\(?2:[[:digit:]]+\\)\\)-\\(?:\\(?3:[[:digit:]]+\\)\\)):(\\(?:\\(?6:[[:digit:]]+\\)\\)-\\(?:\\(?7:[[:digit:]]+\\)\\))\\)\\)?" 1 #1# #2# 2))

But it's more readable as an rx form:

  ((error line-start "STATIC ERROR: " (file-name) ":"
          (or (seq line ":" column (zero-or-one (seq "-" end-column)))
              (seq "(" line ":" column ")" "-"
                   "(" end-line ":" end-column ")"))
          ": " (message) line-end)
   (error line-start "RUNTIME ERROR: " (message) "\n"
          (? "\t" (file-name) ":" ;; first line of the backtrace
             (or (seq line ":" column (zero-or-one (seq "-" end-column)))
                 (seq "(" line ":" column ")" "-"
                      "(" end-line ":" end-column ")")))))

[sbarzowski]

I'm in the process of updating the checker definition for jsonnet

Awesome!

(in general we prefer when tools can report their errors in a machine-friendly format, such as JSON or XML, but IIUC jsonnet only supports human-readable output; correct?)

Currently we only have a human-readable format, admittedly a bit messy one. It wouldn't be hard to support JSON output, especially for go-jsonnet. I proposed it a while back (#366), but nobody seemed to care at the time.

I understand that static errors have a single location and that runtime errors have a stack trace; correct?

Yes.

It would be great if jsonnet could consistently print at least one location for each error, but we'll adjust even if it can't :)

That's something we should definitely fix.