melexis / warnings-plugin

Command-line tool for marking builds with too many warnings as failed. A command-line alternative for https://github.com/jenkinsci/warnings-plugin. Useable with plugin-less CI systems like Gitlab-CI and even Travis-CI
Apache License 2.0
7 stars 2 forks source link

Implement Polyspace checker #140

Open Letme opened 3 months ago

Letme commented 3 months ago

As well as code-quality-report for the GitLab Merge Request GUI.

Here is the output we would need to capture/parse:

Checks statistics: (including internal files)
 - ABS_ADDR => Green :    33, Orange :     0, Red :     0   (100%)
 - COR      => Green :    29, Orange :     3, Red :     0   (91%)
 - IDP      => Green :   419, Orange :    66, Red :     0   (86%)
 - IRV      => Green :    72, Orange :     0, Red :     0   (100%)
 - NIP      => Green :   667, Orange :     0, Red :     0   (100%)
 - NIV      => Green :   338, Orange :    28, Red :     0   (92%)
 - NIVL     => Green :   458, Orange :     3, Red :     0   (99%)
 - OBAI     => Green :    95, Orange :     0, Red :     0   (100%)
 - OVFL     => Green :   450, Orange :    30, Red :     0   (94%)
 - SHF      => Green :   142, Orange :     0, Red :     0   (100%)
 - STD_LIB  => Green :     5, Orange :    13, Red :     0   (28%)
 - ZDV      => Green :     7, Orange :     1, Red :     0   (88%)
TOTAL:    => Green :  2715, Orange :   144, Red :     0   (95%)
Number of NTL : 0
Number of NTC : 0
Number of UNR : 391
Number of functions defined but never called    : 345 / 546
Number of functions called from unreachable code: 40 / 546
Classification of orange checks:
  Orange checks not classified        : 138
  Orange checks from imprecise sources: 6
    - Absolute addresses: 5
    - Extern variables: 1
Classification of orange check sources:
  Extern variables
    - Extern variable io0x28a may be source of 1 orange check
  Absolute addresses
    - Absolute addresses may be source of 5 orange checks
First orange checks to review:
  - Number of orange checks that are path-related issues              : 8
  - Number of orange checks that are bounded input issues             : 0
GUI files generation completed in 7.53real

I believe important output here is: TOTAL: => Green : 2715, Orange : 144, Red : 0 (95%) . My conclusion would be:

JasperCraeghs commented 1 month ago

Polyspace can export its results to a TSV file. If the warnings plugin can parse this TSV file and it contains the location of Polyspace issues in the source code, a Code Quality report can be generated. Would this be worth the extra effort?

Letme commented 1 month ago

I dont know what all is in that tsv file. I think it would help us produce a decent code quality report indeed as a benefit and it would automatically compare it to the job in master, so it could only display a difference.

JasperCraeghs commented 3 weeks ago

The TSV file should contain both Code Prover and Bug Finder results. Let's aim to support both.