coverage data integration

[jmberg]

I've been spending far too much time looking at code coverage reports, and realized that it'd be nice to be able to jump to identifiers, look at different versions, etc. which ultimately means I basically want elixir ... but, with a coverage data overlay.

There's an example lcov/htmlcov report here: http://ltp.sourceforge.net/coverage/lcov/output/index.html

Would it be a complete stretch to add something like that to elixir, presumably by reading the lcov file (very simple machine-readable format) for a given version/tag/... and putting that into the HTML in a fashion similar to the above lcov HTML report?

I have absolutely zero clue how elixir works internally or how the code representation works etc., so throwing out the question before I look into it. Also, no idea if you'd even want such a thing.

Edit: come to think of it, ideally, there would even be different coverage reports for each version, e.g. on v5.9 you might have different coverage reports to overlay, depending on which test suite was run, or which architecture, etc.; there are ways of combining lcov reports so such a "combined" thing could be handled by adding yet another lcov file, which would also be useful.

[jmberg]

OK, looks completely doable, I have a PoC. Just need to override the pygments formatter a bit...

The URL scheme is a bit ugly, but for now I settled on something like

https://elixir.example.com/linux/:cov=asdf/v5.4.3/source

since tags cannot contain ":" in git, that's not ambiguous. That would also allow other extensions to be stuck in there in the future.

[cxw42]

The colon is a good idea for overlays on an existing page. Would it be worth a separate action for source+coverage? E.g.,

https://elixir.bootlin.com/linux/v5.9.13/source/drivers/Kconfig - file
https://elixir.bootlin.com/linux/v5.9.13/coverage/drivers/Kconfig - coverage report

or maybe

https://elixir.bootlin.com/linux/v5.9.13/source:coverage/drivers/Kconfig

with :foo indicating that foo should be added to the given view?

[jmberg]

Hm, yes, that's another good vector for extension.

However, (ideally) we'd want to preserve this selection over a search, e.g. if you go from /linux/v5.9.13/source:coverage/fileA to /linux/v5.9.13/C/ident/foo and then to a new file, we'd still want to be in coverage mode such as /linux/v5.9.13/source:coverage/fileB.

So ... I don't mind adding it at the command point instead of at the tag point, but I don't think it would be something like a "special coverage mode for source", it should remain more a "browse mode". And I'd probably still suggest it should be /linux/v5.9.13/source:cov=coverage-dataset/fileA, so we could extend it with other modes in the future by adding more :xyz=asdf after the command?

Also, another wrinkle here is that the coverage-dataset, and even its availability, is actually dependent on the tag. You might not have the (same) coverage data for each tag. Ideally, we'd preserve the coverage dataset if it exists, but if it doesn't exist we should redirect to the normal page w/o the coverage set.

From a UI point of view, I therefore don't think it matters much where the extra data is added to the URL.

However, from a code POV I think it makes a difference. Adding it at the front was really simple, because that's just the base URL that propagates through everything, and doesn't require changing all the code that generates the links. This is simple and clean. Adding it to the command means (at least I think it does) you have to touch all the places that generate the link, since even the /source/ view will generate links to other /source/ pages, and links to /C/ident/ pages, since it marks up the document. I think it'd be very complex to get that done, and correctly.

[jmberg]

Old PoC code, based on commit 70e868d4bf0d4

poc.txt