search

ShiftLeftSecurity / sast-scan

Scan is a free & Open Source DevSecOps tool for performing static analysis based security testing of your applications and its dependencies. CI and Git friendly.
https://discord.gg/DCNxzaeUpd
Apache License 2.0
796 stars 111 forks source link

Depscan html report not generated #198

Open gitnachogo opened 4 years ago

gitnachogo commented 4 years ago

Hi! I've been integrating sast-scan in my CI/CD pipelines in GitLab, and I've realized the depscan and license-scan html reports are not generated. I have a feeling that the java/JDK version is possibly having to do with this. Before, I used to set the JAVA__HOME: /data/java/jdk1.8.0_102/jre variable and the reports were generated:

 [10:34:22] DEBUG    Depscan and HTML report written to file: /builds/nacho.guisado/helloshiftleft/reports/depscan-report-java.json,         
                     /builds/nacho.guisado/helloshiftleft/reports/depscan-report-java.html 👍                                                
 [10:34:23] DEBUG    License check and HTML report written to file: /builds/nacho.guisado/helloshiftleft/reports/license-java.json,          
                     /builds/nacho.guisado/helloshiftleft/reports/license-java.html 👍

However, I wouldn't like to set the JAVA_HOME as JRE anymore because the mvn package job requires a JDK, and I've let it as it comes. Since then, the .json files are generated, but not the html.

This is the info I've extracted from the scan image (shiftleft/sast-scan):

$ java -version
 openjdk version "11.0.8" 2020-07-14 LTS
 OpenJDK Runtime Environment 18.9 (build 11.0.8+10-LTS)
 OpenJDK 64-Bit Server VM 18.9 (build 11.0.8+10-LTS, mixed mode, sharing)
 $ echo $JAVA_HOME
 /usr/lib/jvm/jre-11-openjdk

Thanks so much in advance!

prabhu commented 4 years ago

Any bom-java.json got created under reports directory?

If your application specifically needs Java 8, you can use the shiftleft/scan-java container image. This has both Java 8 and 11. Then set the environment variable USE_JAVA_8.

If this still doesn't work, run cdxgen tool separately to see if it is able to produce the file.

mkdir -p ${CI_PROJECT_DIR}/reports
npm install -g @appthreat/cdxgen
cdxgen -t java -o ${CI_PROJECT_DIR}/reports/bom-java.json
gitnachogo commented 4 years ago

All of these have been the files generated. Does this help? 

bom-java.json | 147 KB
bom-java.xml | 157 KB
class-report.html | 56.7 KB
class-report.sarif | 54.2 KB
class-report.xml | 82 KB
depscan-report-java.json | 119 KB
inspect-report.findings.json | 78.9 KB
license-java.json | 681 Bytes
scan-full-report.json | 82.4 KB
source-java-report.csv | 77 Bytes
source-java-report.html | 16.5 KB
source-java-report.sarif | 2.57 KB

I don't think there is any problem scanning dependencies because the results are shown in the cli: image

gitnachogo commented 4 years ago

However, I've tried using shiftleft/scan-java with the USE_JAVA_8: "true" and it still does not generate an html for the depscan and license scan. Here's the info the image outputs:

 $ java -version
 openjdk version "1.8.0_265"
 OpenJDK Runtime Environment (build 1.8.0_265-b01)
 OpenJDK 64-Bit Server VM (build 25.265-b01, mixed mode)
 $ echo $JAVA_HOME
 /usr/lib/jvm/jre-1.8.0

Btw, files generated:

bom-java.json | 114 KB
bom-java.xml | 128 KB
class-report.html | 56.7 KB
class-report.sarif | 54.2 KB
class-report.xml | 78.3 KB
depscan-report-java.json | 119 KB
inspect-report.findings.json | 78.3 KB
license-java.json | 681 Bytes
scan-full-report.json | 82.3 KB
source-java-report.csv | 77 Bytes
source-java-report.html | 16.5 KB
source-java-report.sarif | 2.55 KB
prabhu commented 4 years ago

You are no longer seeing message like below?

 [10:34:22] DEBUG    Depscan and HTML report written to file: /builds/nacho.guisado/helloshiftleft/reports/depscan-report-java.json,         
                     /builds/nacho.guisado/helloshiftleft/reports/depscan-report-java.html 👍                                                
 [10:34:23] DEBUG    License check and HTML report written to file: /builds/nacho.guisado/helloshiftleft/reports/license-java.json,          
                     /builds/nacho.guisado/helloshiftleft/reports/license-java.html 👍
gitnachogo commented 4 years ago

No, that message is not shown yet. Only the SAST scan results are displayed after what I've previously shown in the screenshot

prabhu commented 4 years ago

Ok, I think I know what is happening. These lines are no longer getting executed https://github.com/ShiftLeftSecurity/sast-scan/blob/master/lib/executor.py#L231. This could happen if due to some reason dependency and license scan is taking more time and is continuing in the background because of multiprocessing.

gitnachogo commented 4 years ago

I've been testing the same pipelines that worked once (with the other java version), and re-running them I've found that they're now not displaying that message anymore and, of course, not generating the html and sarif files, using the same .gitlab-ci.yml file content. Do you have any idea of how to solve this? Is this because of something in my environment? Could it be fixed from code?

prabhu commented 4 years ago

I will take it as a defect and fix it.

prabhu commented 3 years ago

@gitnachogo Can you take a look at this and send a PR?

gitnachogo commented 3 years ago

@gitnachogo Can you take a look at this and send a PR?

Yes, I'm hands on it this weekend, let's see if I can settle this out

gitnachogo commented 3 years ago

It's getting hard for me to reproduce this issue anymore. I've downloaded and kept the vulnerabilities database locally, and I've ran it using wsl2, but the html is always being generated