checkstyle output

[BrianHicks]

It'd be really nice if elm-format --validate src/ could generate checkstyle-compatible XML. It would go really well with elm-test's junit output. This is totally feasible; I've already written a script to transform from JSON to XML. There's not a lot involved. There are two big steps that would improve support here:

• add --report-format={json|checkstyle} or similar to elm-format so I can deprecate my custom transformation script.
• add line number of violations to the report output. This information is lost right now, so we can only return to the file instead of seeing precisely where the violation occurred. Aside from that, Jenkins' checkstyle plugin will blame the wrong person as having last touched the violating line. IMO this is not useful to begin with, but it should at least be correct.

The first option seems simple enough to add to elm-format. If you're interested, ping me here or privately and I will probably submit a PR when I get time. :)

[avh4]

@ento originally implemented junit output format, but I was hesitant to support an arbitrary set of formats and wanted to try exposing a single output format with a schema that exactly matched the data being output. I was imagining a simple command line tool to read the json and write format x could be published via npm. Does that seem like a bad way to do things? Your CI script probably already uses npm to install elm-format, it could just be a matter of adding elm-format-whatever-format to your npm install, and then changing your script to elm-format --validate src | elm-format-whatever-format out.xml

Reporting line numbers should be a separate issue, and could use some examples of what line(s) should be reported in which circumstances.

[BrianHicks]

Does that seem like a bad way to do things?

My thought: it's not a bad way to do things, but right now it's an inconsistent user experience. The report format should be human-readable or machine readable, and this should be triggered by a flag. So --report-format=human by default, json when needed.

As for the script output, you're probably right! I wrote mine in Python (because easy XML construction) but publishing to node may make more sense.

[lassik]

(I'll use this issue to discuss machine-readable error output in general since this is the most appropriate issue I found and didn't want to open yet another one, hope it's ok.)

Machine-readable error output ought to be very useful, but IMHO largely if it's in a standard (JSON or XML) format emitted by many formatters, so plugins that support several formatters could use the same code to parse the output from each of them and present it identically. All formatters emit mostly compiler-style error messages anyway:

• filename
• line number
• bad code snippet
• error or warning code
• human-readable message

If elm-format wants to include some non-standard formatter-specific information in the output, that would probably work fine, since both JSON- and XML-based schemas are quite easy to extend. IMHO, the main thing would be for the usual fare (line X had error/warning Y) to be standardized among formatters.

The file format itself would probably be quite trivial, but getting everyone on board to implement it would be a major task. I would just start by asking a few other formatter authors if they are interested in working together on it.

I don't know whether the checkstyle format would fit the bill. rustfmt supports it. I would personally love to have a JSON format instead of XML, but I don't know of any established ones. If some compiler (e.g. LLVM?) or linter has JSON error output, we could probably adopt that schema and use it for code formatters with few (if any) changes. Otherwise, at least checkstyle is widely used. By quick googling, it doesn't seem to have a JSON schema, only XML.

In the meantime, I have found the default human-readable error output of formatters to be very reasonable for presenting to users directly (I'm doing that in Emacs). The only thing is that ANSI color presents a problem since it's not going to the terminal.

[lassik]

Here's yet another format: the Vim quickfix error format