search

jeremylong / DependencyCheck

OWASP dependency-check is a software composition analysis utility that detects publicly disclosed vulnerabilities in application dependencies.
https://owasp.org/www-project-dependency-check/
Apache License 2.0
6.44k stars 1.28k forks source link

Owasp Dependency check 7.4.4 is crashing automatically #5288

Closed vignesh-gupta-xom closed 1 year ago

vignesh-gupta-xom commented 1 year ago

Describe the bug Dependency check is failing automatically

Version of dependency-check used The problem occurs using version 7.4.4 of the gradle plugin ('org.owasp:dependency-check-gradle:7.4.4')

Log file `considered a test configuration: true

No longer waiting for event queue to finish: Pooled Cache Event Queue Working = true Alive = false Empty = true Queue Size = 0 Queue Capacity = 2147483647 Pool Size = 0 Maximum Pool Size = 150 Region [NODEAUDIT] Saving keys to: NODEAUDIT, key count: 0 Region [NODEAUDIT] Finished saving keys. Region [NODEAUDIT] Shutdown complete. No longer waiting for event queue to finish: Pooled Cache Event Queue Working = true Alive = false Empty = true Queue Size = 0 Queue Capacity = 2147483647 Pool Size = 0 Maximum Pool Size = 150 Region [CENTRAL] Saving keys to: CENTRAL, key count: 0 Region [CENTRAL] Finished saving keys. Region [CENTRAL] Shutdown complete. No longer waiting for event queue to finish: Pooled Cache Event Queue Working = true Alive = false Empty = true Queue Size = 0 Queue Capacity = 2147483647 Pool Size = 0 Maximum Pool Size = 150 Region [POM] Saving keys to: POM, key count: 0 Region [POM] Finished saving keys. Region [POM] Shutdown complete.

Task :dependencyCheckAggregate FAILED In dispose, destroying event queue. In DISPOSE, [NODEAUDIT] disposing of memory cache. Memory Cache dispose called. In DISPOSE, [CENTRAL] fromRemote [false] In DISPOSE, [CENTRAL] auxiliary [CENTRAL] In DISPOSE, [CENTRAL] put 0 into auxiliary CENTRAL In dispose, destroying event queue. In DISPOSE, [CENTRAL] disposing of memory cache. Memory Cache dispose called. In DISPOSE, [POM] fromRemote [false] In DISPOSE, [POM] auxiliary [POM] In DISPOSE, [POM] put 0 into auxiliary POM In dispose, destroying event queue. In DISPOSE, [POM] disposing of memory cache. Memory Cache dispose called. :dependencyCheckAggregate (Thread[Daemon worker,5,main]) completed. Took 12.156 secs.

FAILURE: Build failed with an exception.

How did it happen We were using 6.0.2 but then there was [this issue] (https://github.com/jeremylong/DependencyCheck/issues/5220) and we changed the version to solve that but now this issue is the issue we are failing.

jeremylong commented 1 year ago

Try running ./gradlew dependencyCheckPurge and then re-running ./gradlew dependencyCheckAggregate. If you are still seeing an issue please provide the configuration used (i.e. the dependencyCheck section of your build.gradle. Additionally, the detailed log (.i.e., ./gradlew dependencyCheckAggregate --debug may highlight what the issue is.

vignesh-gupta-xom commented 1 year ago

Hello @jeremylong , Thanks for responding. Here is is configuration: `dependencyCheck { failBuildOnCVSS = 5 format = 'ALL' suppressionFile = 'config/dependencyCheck/suppressions.xml' outputDirectory = file("$project.buildDir/reports/dependencycheck") analyzers { ossIndex { enabled = false } retirejs { retireJsUrl = 'https://raw.githubusercontent.com/RetireJS/retire.js/master/repository/jsrepository.json' } }

    cve {
        urlBase='https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-%d.json.gz'
        urlModified='https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz'
    }
}

` I cannot run debug as it's failing in server.

aikebah commented 1 year ago

` I cannot run debug as it's failing in server.

Surely you or one of your peers would be able to update your CI to run the job in gradle --debug mode instead of regular mode? --debug is just increasing the verbosity of the logging of gradle to debug level.

vignesh-gupta-xom commented 1 year ago

Hello there, it's working in my local but issue in Openshift only. So, I made change and using 7.4.1 now. And in that

Instead of 

Finished Dependency Merging Analyzer (0 seconds)
Finished Version Filter Analyzer (0 seconds)
Finished Hint Analyzer (0 seconds)
Created CPE Index (2 seconds)
Finished CPE Analyzer (6 seconds)
Finished False Positive Analyzer (0 seconds)
Finished NVD CVE Analyzer (0 seconds)
Finished RetireJS Analyzer (2 seconds)
Finished Vulnerability Suppression Analyzer (0 seconds)
Finished Dependency Bundling Analyzer (0 seconds)
Analysis Complete (16 seconds)
-> Generating report for project giar-customer-buyback-service
Found 0 vulnerabilities in project giar-customer-buyback-service
Element event queue destroyed: org.apache.commons.jcs.engine.control.event.ElementEventQueue@4e18f096
In DISPOSE, [NODEAUDIT] fromRemote [false]
In DISPOSE, [NODEAUDIT] auxiliary [NODEAUDIT]
In DISPOSE, [NODEAUDIT] put 0 into auxiliary NODEAUDIT

I'm getting Element event queue destroyed 

Finished Dependency Merging Analyzer (0 seconds)
Finished Version Filter Analyzer (0 seconds)
Finished Hint Analyzer (0 seconds)
Created CPE Index (1 seconds)
Finished CPE Analyzer (7 seconds)
Finished False Positive Analyzer (0 seconds)
Finished NVD CVE Analyzer (0 seconds)
Exception occurred initializing RetireJS Analyzer.
Finished Vulnerability Suppression Analyzer (0 seconds)
Finished Dependency Bundling Analyzer (0 seconds)
Analysis Complete (14 seconds)
-> Element event queue destroyed: org.apache.commons.jcs.engine.control.event.ElementEventQueue@3df6e509
In DISPOSE, [NODEAUDIT] fromRemote [false]
In DISPOSE, [NODEAUDIT] auxiliary [NODEAUDIT]
In DISPOSE, [NODEAUDIT] put 0 into auxiliary NODEAUDIT

No longer waiting for event queue to finish: Pooled Cache Event Queue
vignesh-gupta-xom commented 1 year ago

TO add on. We tried using --debug but it throws an error security info leak. image

jeremylong commented 1 year ago

If you are caching the data directory in Jenkins (which is recommended) - you may need to setup the job to run dependencyCheckPurge before running dependencyCheckAnalyze as a one time resolution to the problem. Then you can remove the purge to improve performance.

vignesh-gupta-xom commented 1 year ago

We have not cached temp. for some reasons. But dependencyCheckPurge is running everytime and is done successfully. I ran the same thing with sttacktrace and got this as error.

Caused by: org.owasp.dependencycheck.exception.ExceptionCollection: One or more exceptions occurred during analysis:
    InitializationException: Failed to initialize the RetireJS repo: `/tmp/dctemp54977843-e9d0-4170-92af-ce881ae11ba0/jsrepository.json` appears to be malformed. Please delete the file or run the dependency-check purge command and re-try running dependency-check.
        caused by JSONException: Value <!DOCTYPE of type java.lang.String cannot be converted to JSONObject
    at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:665)

But when opening the retire js link then getting the correct json. Any known solution?

aikebah commented 1 year ago

@vignesh-gupta Your Jenkins is likely getting a proxy "access has been denied by your network/proxy team" HTML page instead of the RetireJS json file

aikebah commented 1 year ago

Disable the RetireJSAnalyzer,

open up access to the RetireJS json on your proxy infrastructure

or host a local copy of the RetireJS json in your datacenter and setup some way to periodically synchronize it with the official RetireJS json to ensure you're not too far behind on Javascript vulnerabilities and set the retireJS URL to your datacenter mirror in the gradle configuration

vignesh-gupta-xom commented 1 year ago

@aikebah I got your point but using a file that updates daily is throwing an error:


InitializationException: Unexpected Exception
                caused by IllegalArgumentException: URI has an authority component
        at org.owasp.dependencycheck.Engine.analyzeDependencies(Engine.java:665)
        at org.owasp.dependencycheck.Engine$analyzeDependencies$0.call(Unknown Source)
        at org.owasp.dependencycheck.gradle.tasks.AbstractAnalyze.analyze(AbstractAnalyze.groovy:90)
        ... 185 more

I'm using dependency-check-gradle:7.1.1

And My config for retire json is


 dependencyCheck {
        failBuildOnCVSS = 5
        format = 'ALL'
        suppressionFile = 'config/dependencyCheck/suppressions.xml'
        outputDirectory = file("$project.buildDir/reports/dependencycheck")
        analyzers {
            ossIndex {
                enabled = false
            }
            retirejs {
                retireJsUrl = 'file://owasp/cache/jsrepository-manual.json'
            }
        }

        cve {
            urlBase='https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-%d.json.gz'
            urlModified='https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz'
        }
    }
vignesh-gupta-xom commented 1 year ago

Any Idea how to pass the file destination for retirejs in 7.1.1 version. I've tried file://owasp/cache/jsrepository-manual.json and /owasp/cache/jsrepository-manual.json. Both throws error. One says url malformed and other says mention the protocol.

And If possible, please provide the doc link for version 7.1.1. This doc is for 8.0.0 .

jeremylong commented 1 year ago

try using three slashes:

retireJsUrl = 'file:///owasp/cache/jsrepository-manual.json'
vignesh-gupta-xom commented 1 year ago

Hey @jeremylong Thanks for assisting! The above solution worked. Apologies for not responding back :sad:.