search

MarketSquare / robotframework-robocop

Tool for static code analysis of Robot Framework language
Apache License 2.0
189 stars 38 forks source link

Show report in HTML #884

Open tuyentran81 opened 1 year ago

tuyentran81 commented 1 year ago

Currently, the report after scanning is in text file. It's hard to read it. I would like to have the report in HTML format after running report.

mnojek commented 1 year ago

Hey @tuyentran81 and thanks for the issue. We are currently investigating if we can add this and how. We will let you know if we find a good way of visualizing the Robocop findings.

bollwyvl commented 1 year ago

A SARIF report can be transformed into HTML with sarif-tools, which is conveniently implemented in python.

However, as it depends on matplotlib, this is probably not something that should be a hard dependency, but a robotframework-robocop[html] might be reasonable "extra"...

mnojek commented 1 year ago

Thanks @bollwyvl for the suggestion! I tried this out and while this is totally doable, the output HTML is not something I would call "nice and readable".

I think the main idea behind this issue is that we want to have a custom HTML report that is the best visual representation of the code issues fund by Robocop, which means it should be adjusted to the type of this document.

But the sarif-tools is a good candidate for an intermediate solution.

bhirsz commented 1 year ago

I have also tested it and it's how it looks like (from sarif html command): image

I guess we can decide on one of the two kind of reports: summary or detailed report. Sarif is summary kind (where we put the information, graphs from the whole run). The detailed could be for example coverage report:

image

We could display Robocop errors instead (with small summary at top of the file). I'm leaning more forward the second kind. It's definietely more work but we can get inspired by coverage package and reuse their code (open source code rocks ;)). They're using Jinja templating: https://github.com/nedbat/coveragepy/tree/master/coverage/htmlfiles . WDYT?

bollwyvl commented 1 year ago

Yep, sarif-tools isn't great at what it does, and the pie chart is... not that useful, given the weight of the dependency. But sure, "HTML reporting" means a lot of different things, and pinning it down the meaning more precisely with what isn't wanted is fine. Once wading into HTML, there are a lot more (sometimes conflicting) concerns than console/data output: accessibility, searchability, interlinking, file size, branding, etc.

Funny coverage.py was mentioned, as I had recently done some experiments writing a plugin to handle .r(obot|esource) files during a (pa|r)obot run, akin to the django one... which 6.1 broke :crying_cat_face:. But this kinda points to a bunch of correlated concerns: one could imagine a "source first" annotated view of different kinds of data: linter outputs, test result fails/counts/times, profiler output, VCS blame (which sarif-tools kind hints at) and correlations between them. Ideally such a system could span multiple languages, and be very easy to integrate novel forms of "simple" (e.g. well-structured JSON or XML) data.