Don't use XSLT in report generator

[JukkaL]

XSLT is not a widely used technology and pretty complex (and I don't want to learn it). We shouldn't rely on it in the report generator. Some reasonable alternatives include straight XML manipulation or JSON.

[o11c]

The thought behind my use of XSLT is: the primary report is now the machine-readable XML file. It is up to the user to decide how they want to view that - say, combine a python-cov report with a mypy report in a single report. For that motivating example, it is impossible for mypy to produce a meaningful HTML file, so the fact that XML files can be viewed directly with XSLT is a huge bonus (of course, nothing, but I think it's very valuable that we ship an example XSLT file).

Plus (ignoring a little of the txt hackery which I did just to prove how flexible XSLT is, and some percentage calculations which we could precompute if we really had to), all the XSLT we use is pretty simple "get this part of the XML file, put it here in the output HTML file".

Besides, I'm way more active than you are anyway.

It's true that XSLT got less popular with the rise of languages like Markdown, but those are generally not machine-readable with arbitrary attributes like XML is.

And XSLT actually works quite similarly to CSS, which lots of people know. (Possibly some CSS guru could even display the XML files directly without XSLT, but that doesn't mean it's a good idea).

[JukkaL]

I agree that technically XSLT does the job. It's just that I don't like it :-P

Let's wait for more feedback -- maybe people are happy with it. First we should document how the report generator works, though.

[JukkaL]

Added #912 for documenting the report generator.

[refi64]

Is there a reason you can't just use JSON? It's much easier (and faster) to parse.

[o11c]

If you're parsing it yourself, you're doing it wrong, use a library. You should treat XML (or JSON) as an opaque format that is only readable by special tools.

JSON has a number of problems (limited unicode support, no integers, no standard schema tools, very poor tooling in general) that XML does not have.

[refi64]

If you're parsing it yourself, you're doing it wrong, use a library.

Exactly. I know of several languages that have built-in JSON parsers but not XML parsers. Writing an efficient XML parser is hard.

limited unicode support

...yeah, you got me on this one. ;)

no integers

Huh? Aren't normal numbers good enough?

no standard schema tools

Not quite sure what this means...

very poor tooling in general

Does JSON even need that much tooling to begin with, though?

[o11c]

If you're parsing it yourself, you're doing it wrong, use a library.

Exactly. I know of several languages that have built-in JSON parsers but not XML parsers. Writing an efficient XML parser is hard.

Languages, just like end developers, shouldn't be writing their own JSON or XML parsers, but linking to a C implementation. JSON is merely a more tempting target.

no integers

Huh? Aren't normal numbers good enough?

No. Floats are evil.

no standard schema tools

Not quite sure what this means...

When mypy produces a report, it verifies that the XML nodes follow a certain structure. Besides preventing bugs in the generator, this allows tools that consume mypy XML files to automatically verify that they really are handling everything correctly.

While there are a number of JSON schema tools, none is anywhere near even a de-facto standard.

very poor tooling in general

Does JSON even need that much tooling to begin with, though?

A data language is useless without tooling.

Show me a way to combine a python-cov report with a mypy report for the same file. It's easily possible with XSLT.

[refi64]

Languages, just like end developers, shouldn't be writing their own JSON or XML parsers, but linking to a C implementation. JSON is merely a more tempting target.

...a C implementation which is almost always more painful. Not even Expat supports the entirety of XML. The parsers that support everything are huge and slow as heck.

No. Floats are evil.

Why? When doing arithmetic operations with integers stored as floats, the precision is perfectly fine.

[o11c]

...a C implementation which is almost always more painful. Not even Expat supports the entirety of XML. The parsers that support everything are huge and slow as heck.

lxml, which I used in mypy, is a great example of how to write a non-painful wrapper over a C library.

And libxml2 comparse very favorably for performance, see http://lxml.de/performance.html

[refi64]

You forgot the part where installing lxml on Windows is horribly broken.

[jhance]

Using XSLT requires me to write XML therefore I will never use it

[o11c]

Using XSLT requires me to write XML therefore I will never use it

That's what I said about XML until I used XSLT. In my opinion, XSLT tooling alone forgives all the human-unfriendliness from the rest of XML.

[JukkaL]

It seems like we have a consensus forming... XSLT is losing, unless it can gather some more support.

[ilinum]

I think we can use jinja2 instead of XSLT. It is more common than XSLT and so should be easier to use.

[gvanrossum]

Do we still want to get rid of xslt? Then I think the jinja2 idea is a fine one. Would we completely lose the lxml dependency in favor of jinja2?

[JukkaL]

It would still be nice to get rid of the lxml dependency and I like the idea of using jinja2 instead. Jinja2 is much better known than XSLT in the Python community and it's also easier to learn, in my opinion.

[ilinum]

I think we can get rid of xslt if we switch to another jinja2.

Coverage.py has their own template renderer that uses Django/Jinja2-like syntax. I wonder why they chose that over standard jinja2.

[gvanrossum]

I think the comment about the aosa 500 lines book http://aosabook.org/en/500L/a-template-engine.html says it all -- it was an educational project. Also, it takes away an external dependency, but that's not important for us. Let's go with actual jinja2.

On Thu, Nov 2, 2017 at 8:35 AM, Svyatoslav Ilinskiy < notifications@github.com> wrote:

I think we can get rid of xslt if we switch to another jinja2.

Coverage.py has their own https://bitbucket.org/ned/coveragepy/src/9cf891202041c5cbafc19e8c2d5467db77100066/coverage/templite.py?at=default&fileviewer=file-view-default template renderer that uses Django/Jinja2-like syntax. I wonder why they chose that over standard jinja2.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/python/mypy/issues/909#issuecomment-341461021, or mute the thread https://github.com/notifications/unsubscribe-auth/ACwrMnu6ya9vQI7xil_HC4io3xJ85ovRks5syeE-gaJpZM4GM1H_ .

-- --Guido van Rossum (python.org/~guido)