Summary in Reports

[Jeeppler]

It would be very helpful to have a summary in the report. Currently, there is no summary of the findings found. Having a summary in the report would allow a human to get a quick overview of the findings in the report, without having to manually count the number of findings. Counting the number of findings is time consuming, especially in large projects with many findings.

Furthermore, SecHub reports should be readable without requiring any additional tools. Including a summary in the JSON and HTML report would allow that.

Most important to know is the number of findings per scan type and how many findings red, yellow and green findings were found. For example:

Code Scan:

• Total: 18
• Red: 0
• Yellow: 1
• Green: 17

Infra Scan:

• Total: 11
• Red: 0
• Yellow: 4
• Green: 7

In addition, the report could include a more detailed overview of the finding categories. For example:

Code Scan

Medium:

• CWE-36 - Absolute Path Traversal (Count: 1)

Low:

• CWE-379 - Creation of Tempdir... (Count: 5)
• CWE-259 - Use Of Hardcoded Password (Count: 3)

The mock-up below shows how a summary could look like in the HTML report.

The following JSON shows the how a summary could look like in the JSON report.

{
   "jobUUID": "b2832de0-b9ce-4daa-b7b8-5fd89b9f62c4",
   "result": {
      "count": 27,
      "metadata": {
        "summary": {
            "codeScan": {
                "total": 18,
                "red": 0,
                "yellow": 1,
                "green": 17,
                "details": {
                    "medium": [
                        {
                            "cweId": 36,
                            "name": "CWE-36 - Absolute Path Traversal",
                            "count": 1
                        }
                    ],
                    "low": [
                        {
                            "cweId": 379,
                            "name": "CWE-379 - Creation of Tempdir...",
                            "count": 5, 
                        }
                        {   
                            "cweId": 259,
                            "name": "Use Of Hardcoded Password",
                            "count": 3, 
                        }
                    ]
                }
            },
            "infraScan": {
                "total": 11,
                "red": 0,
                "yellow": 4,
                "green": 7,
                "details": {
                    ...
                }
            },
        }
      },
      "findings": [
         {
            "id": 220,
            "name": "Portability Flaw In File Separator",
            "severity": "INFO",
            "code": {
               "location": "handlers/Authentication.java",
               "line": 162,
               "column": 69,
          ...

---

^{Jeremias Eppler jeremias.eppler@daimler.com, Daimler TSS GmbH, imprint}

[sven-dmlr]

Good Ideas :+1:

From a reporting perspective, it would be helpful if the summary data is easily accessible in the database. (not only stored as a large text field)

[Jeeppler]

@sven-dmlr it should be stored in the database to be able to do reporting.

[de-jcup]

HTML report

• if multiple types, than overview with a hyperlink inside the document
• just representation changes as wished in figures
• we group different scan types (link in overview helps to jump into)

JSON reporting

• will be kept as is

Data storage

• overview data (how man low findings, high findings,etc. duplicated etc. )
• sereco could store these information into a special metadata database/table later (other issue)

[Jeeppler]

It would be nice to have the summary in the JSON report as well. The JSON report is the default report and it is tedious to analyze the JSON report by the "search" feature.

[zigfridus]

@Jeeppler assign me to this issue please.

[Jeeppler]

@zigfridus thanks for taking this on.

[zigfridus]

Hello @Jeeppler I'm trying to scan a testcase file and generate a summary HTML report with help of Sechub. I started Sechub server in the IDE dubugger. But after scanning Cechub shows me the warning: no product results for [NESSUS, NETSPARKER, CHECKMARX, PDS_CODESCAN, PDS_WEBSCAN, PDS_INFRASCAN, PDS_LICENSESCAN, PDS_SECRETSCAN] found, will return an empty sereco JSON as result! and generates empty report. I use SecHub Quickstart Guide for this scanning. Maybe there should be some kind of preconfiguration of Sechub for give it possibility to scan. Could you please give me instruction for it? Thanks

[Jeeppler]

@zigfridus I can confirm that is a bug. I tried to replicate it and get an empty report back as well.

[Jeeppler]

I created #2071 to track the problem.

[Jeeppler]

Another way to test it: start a pds-solution, like PMD, in a container and connect SecHub to it.

[ SecHub in your IDE ]  <-----> [ PMD+PDS ]

Next, you have to setup a project, executor etc. You can find scripts in sechub-solution/setup-pds to help you with the setup. For PMD the script is called: setup-pmd.sh. You have to adjust the default values to the integration test mode.

JVM arguments for the integration test mode:

-Dspring.profiles.active=mocked_products,h2,integrationtest
-Dsechub.server.debug=true
-Dsechub.storage.sharedvolume.upload.dir=/home/user/.sechub/sharedvolume
-Dsechub.targettype.detection.intranet.hostname.endswith=intranet.example.org
-Dsechub.config.trigger.nextjob.initialdelay=0

UPDATE: You have to use the dev,real_products,mocked_notifications,h2 profiles. Otherwise you are unable to use external PDS solutions.

[zigfridus]

@Jeeppler I would like to ask about another way you mentioned before. Do I have to start a sechub or a sechub-pds server in the IDE?

[Jeeppler]

In both cases you have to start only SecHub in the IDE.

Option A) Start SecHub in integration test mode -> get a mock report back. Problem, currently it does not work: #2071. Option B) Start SecHub with the real_products profile and start a PDS-Solution. Configure SecHub to use the PDS-solution with the help of scripts in sechub-solution/setup-pds -> scan real projects and get real results back.

[zigfridus]

@Jeeppler There are multiple tasks in this issue. The tasks from you, @de-jcup and @sven-dmlr. I would like to ask you to streamline these tasks and specify with what task it's better to start. Thanks

[Jeeppler]

@zigfridus I will refine it with @de-jcup, @sven-dmlr.

[de-jcup]

We should fix the mentioned parts. But this issue can be done easier and faster with next steps described:

Report development without server starts

Remark @zigfridus The next step will work after #2124 has been merged (we had this already, but there was bug #2125 which is fixed by this merge)

Development changes in HTML report

• Change CSS inside scanresult.css
• Change Template content in scanresult.html and included fragements

Generate the output

• Just execute ThymeLeafHTMLReportingTest with
  • ENV variable SECHUB_KEEP_TEMPFILES to true
  • ENV variable SECHUB_AUTO_GENERATE_CSS_FRAGMENTS_ON_HTML_TESTS to true

Check

Look at the output file paths in console and open one of the mentioned html files with your browser and inspect the result. The path will not change, so after changing + restarting unit tests you can simply refresh your browser tab.

This should increase the report development very much.

[Jeeppler]

@zigfridus

We agreed on the following:

• A summary is added to the JSON report format in the meta data section
• A summary is added to the HTML report format on top of the document

Persisting the meta data information in the DB is not part of this issue. This will be implemented as part of a different issue.

JSON report:

{
   "jobUUID": "b2832de0-b9ce-4daa-b7b8-5fd89b9f62c4",
   "result": {
      "count": 27,
      "metaData": {
        "summary": {
            "codeScan": {
                "total": 18,
                "red": 0,
                "yellow": 1,
                "green": 17,
                "details": {
                    "medium": [
                        {
                            "cweId": 36,
                            "name": "CWE-36 - Absolute Path Traversal",
                            "count": 1
                        }
                    ],
                    "low": [
                        {
                            "cweId": 379,
                            "name": "CWE-379 - Creation of Tempdir...",
                            "count": 5, 
                        }
                        {   
                            "cweId": 259,
                            "name": "Use Of Hardcoded Password",
                            "count": 3, 
                        }
                    ]
                }
            },
            "infraScan": {
                "total": 11,
                "red": 0,
                "yellow": 4,
                "green": 7,
                "details": {
                    ...
                }
            },
        }
      },
      "findings": [
         {
            "id": 220,
            "name": "Portability Flaw In File Separator",
            "severity": "INFO",
            "code": {
               "location": "handlers/Authentication.java",
               "line": 162,
               "column": 69,
          ...

HTML Report:

The image below is a mock, please improve the result while implementing it.

In the HTML report the findings will be visually grouped. The first criteria for grouping is the severity (red, yellow, green) and second is the scan type. The structure is shown in the mock, but just to recap it will look like this:

Red

• scan type 1 (e. g. code scan)
• scan type 2 (e. g. secret scan)

Yellow

• scan type 1 (e. g. code scan)
• scan type 2 (e. g. secret scan)

Green

• scan type 1 (e. g. code scan)
• scan type 2 (e. g. secret scan)

In addition, it should be possible to click on the summary of the scan type and severity and jump to all the findings with that severity + scan type:

For example:

Code Scan
  Total: 12
  Red: 3 -> click -> jump to Red + code scan
  Yellow: 2 -> click -> jump to Yellow + code scan
  Green: 7 -> click -> jump to Green + code scan

The jump should be realized with an document internal link: Linking to an element on the same page.

This task will require changes to the representation of the HTML and JSON report. It will not require changes to the actual sereco data model or database model.

Furthermore, keep in mind, we do not want to use JavaScript in the HTML report and keep accessibility in mind when you do the changes. Long term the reports should become fully accessible for all people (with and without disabilities). Accessibility and usability are more important to us, then "looking fancy".

[zigfridus]

@Jeeppler Thank you for the detailed explanation.

[zigfridus]

@Jeeppler Where I can download a source code with Medium and High severity of vulnerability for testing? Thanks

[zigfridus]

@Jeeppler I've found the solution already. Sorry for disturbing you.

[zigfridus]

@Jeeppler I created draft PR. It only creates Summary table in html. Please check if this is what you expected to see. Thanks

[Jeeppler]

@zigfridus I reviewd it.

[zigfridus]

@Jeeppler Thank you

[dAnjou]

Hi, it'd be nice to have info in the report that indicates what has been scanned. Currently, the report for a successful job is quite empty, and you can't tell whether that's because there are actually no findings or whether nothing has been scanned:

{
   "result": {
      "count": 0,
      "findings": []
   },
   "reportVersion": "1.0",
   "jobUUID": "930ea58e-f220-437f-a468-0046543c1573",
   "trafficLight": "GREEN",
   "messages": [],
   "status": "SUCCESS"
}

[Jeeppler]

@dAnjou thanks. However, your comment does not belong in this issue. Please create a new issue or get in contact with @sven-dmlr or @haerter-tss.

[de-jcup]

Attached some examples how the final reports (HTML/JSON) looks like.

sechub-report-examples.zip