search

jeremylong / DependencyCheck

OWASP dependency-check is a software composition analysis utility that detects publicly disclosed vulnerabilities in application dependencies.
https://owasp.org/www-project-dependency-check/
Apache License 2.0
6.3k stars 1.26k forks source link

Support for ServerID to support nexus raw proxy repositories with authentication #2388

Open G-Ork opened 4 years ago

G-Ork commented 4 years ago

It would be nice to support or use std. maven features for downloading http based content so basic features like authentication via server config in settings.xml would be supported.

The clients do not have a direct internet connection. I like to use the nexus raw proxy repo to cache the data. Due the lack of support for http authentication i've to give anonymous access for the nexus repository to use nexus as cache for the cve data. I do not like to give anon access to that repo.

jeremylong commented 4 years ago

If credentails are present in the nexus server specified ODC will use basic auth. See

https://github.com/jeremylong/DependencyCheck/blob/9257510b87bd847947be2a9987a235faf96e499c/core/src/main/java/org/owasp/dependencycheck/data/nexus/NexusSearch.java#L114

https://github.com/jeremylong/DependencyCheck/blob/9257510b87bd847947be2a9987a235faf96e499c/core/src/main/java/org/owasp/dependencycheck/data/nexus/NexusSearch.java#L211-L223

G-Ork commented 4 years ago

Its an different use case. I use a nexus raw proxy repo to serve the cve data. The common place to put authentication information would be settings.xml. The plugin do not pick up from there in this case. I do not like to put the credentials into the pom.

Stacktrace

[INFO] Generating "dependency-check:aggregate" report --- dependency-check-maven:5.2.4:aggregate
[INFO] Checking for updates
[ERROR] Error retrieving https://nexus3/repository/nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta; received response code 401.
[ERROR] Unable to download meta file: https://nexus3/repository/nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta
org.owasp.dependencycheck.data.update.exception.UpdateException: Unable to download meta file: https://nexus3/repository/nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.getMetaFile (NvdCveUpdater.java:347)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.getUpdatesNeeded (NvdCveUpdater.java:385)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.update (NvdCveUpdater.java:122)
    at org.owasp.dependencycheck.Engine.doUpdates (Engine.java:922)
    at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase (Engine.java:723)
    at org.owasp.dependencycheck.Engine.analyzeDependencies (Engine.java:653)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.runCheck (BaseDependencyCheckMojo.java:1403)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.generate (BaseDependencyCheckMojo.java:866)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.generate (BaseDependencyCheckMojo.java:819)
    at org.apache.maven.plugins.site.render.ReportDocumentRenderer.renderDocument (ReportDocumentRenderer.java:239)
    at org.apache.maven.doxia.siterenderer.DefaultSiteRenderer.render (DefaultSiteRenderer.java:349)
    at org.apache.maven.plugins.site.render.SiteMojo.renderLocale (SiteMojo.java:198)
    at org.apache.maven.plugins.site.render.SiteMojo.execute (SiteMojo.java:147)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (Unknown Source)
    at java.lang.reflect.Method.invoke (Unknown Source)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Download failed, unable to retrieve 'https://nexus3/repository/nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta'
    at org.owasp.dependencycheck.utils.Downloader.fetchContent (Downloader.java:115)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.getMetaFile (NvdCveUpdater.java:340)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.getUpdatesNeeded (NvdCveUpdater.java:385)
    at org.owasp.dependencycheck.data.update.NvdCveUpdater.update (NvdCveUpdater.java:122)
    at org.owasp.dependencycheck.Engine.doUpdates (Engine.java:922)
    at org.owasp.dependencycheck.Engine.initializeAndUpdateDatabase (Engine.java:723)
    at org.owasp.dependencycheck.Engine.analyzeDependencies (Engine.java:653)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.runCheck (BaseDependencyCheckMojo.java:1403)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.generate (BaseDependencyCheckMojo.java:866)
    at org.owasp.dependencycheck.maven.BaseDependencyCheckMojo.generate (BaseDependencyCheckMojo.java:819)
    at org.apache.maven.plugins.site.render.ReportDocumentRenderer.renderDocument (ReportDocumentRenderer.java:239)
    at org.apache.maven.doxia.siterenderer.DefaultSiteRenderer.render (DefaultSiteRenderer.java:349)
    at org.apache.maven.plugins.site.render.SiteMojo.renderLocale (SiteMojo.java:198)
    at org.apache.maven.plugins.site.render.SiteMojo.execute (SiteMojo.java:147)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (Unknown Source)
    at java.lang.reflect.Method.invoke (Unknown Source)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)`

Pom-Fragment

<plugin>
    <groupId>org.owasp</groupId>
    <artifactId>dependency-check-maven</artifactId>
    <version>5.2.4</version>

    <reportSets>
        <reportSet>
            <reports>
                <report>aggregate</report>
            </reports>
        </reportSet>
    </reportSets>

    <configuration>
        <serverId>nexus3</serverId>
                 <cveUrlModified>https://nexus3/repository/nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz</cveUrlModified>
                 <cveUrlBase>https://nexus3/repository/nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-%d.json.gz</cveUrlBase>

                 <retireJsAnalyzerEnabled>false</retireJsAnalyzerEnabled>
        <centralAnalyzerEnabled>false</centralAnalyzerEnabled>
    </configuration>
</plugin>

settings.xml (example)

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
                      http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <servers>
    <server>
      <id>nexus3</id>
      <username>username</username>
      <password>{ENCPWD=}</password>
    </server>
jeremylong commented 4 years ago

Please consider hosting the NVD content on a different host.

G-Ork commented 4 years ago

You say i should not use nexus just because the current state of the plugin do not support simple authentication? Do you mind, that it work without auth like a charm?

Distribute a proxy with url filtering isn't what I've got in mind just for some missing lines of code at the right place. Need help or accept an pull request if I'll fix it?

jeremylong commented 4 years ago

PRs are more than welcome.

alan-czajkowski commented 1 year ago

I'm in a similar situation where I have no choice but to raw proxy all of the URLs: pom.xml:

<configuration>
  <cveUrlModified>https://internal-hostname/repository/proxy-raw-nist-nvd/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz</cveUrlModified>
  <cveUrlBase>https://internal-hostname/repository/proxy-raw-nist-nvd/feeds/json/cve/1.1/nvdcve-1.1-%d.json.gz</cveUrlBase>
  <knownExploitedUrl>https://internal-hostname/repository/proxy-raw-cisa/sites/default/files/feeds/known_exploited_vulnerabilities.json</knownExploitedUrl>
  ...
</configuration>

how do I configure the server ID in the plugin's <configuration> for the credentials?

this does not work: pom.xml:

<configuration>
  ...
  <serverId>internal-hostname</serverId>
  <cveServerId>internal-hostname</cveServerId>
</configuration>

settings.xml:

...

  <servers>

    <server>
      <id>internal-hostname</id>
      <username>...</username>
      <password>...</password>
    </server>

    ...

  </servers>

  ...
alan-czajkowski commented 1 year ago

@jeremylong @G-Ork do you have any recommendations on what I'm doing above?

G-Ork commented 1 year ago

@alan-czajkowski In fact using a raw proxy (nexus terminology) is what i did to overcome that problem. I was in lack of time to provide a PR for the plugin. I was able to sell it to the CTO after generically putting a proxy in front of nexus breaking up SSL and do virus checks. After that the security risk using a raw proxy is not that big.

But it would be indeed more easy if the plugin using the http(s) transport mechanisms in maven. Just no extra argues necessary.

alan-czajkowski commented 1 year ago

@G-Ork my issue is that I need to connect to Nexus (all of those https://internal-hostname/repository/proxy-raw... URLs) and it is mandatory that access to my Nexus uses credentials to access those URLs, so I need to use settings.xml with <server> entry ... does anybody know how I can do this with this plugin? cc: @jeremylong

G-Ork commented 1 year ago

@alan-czajkowski Look at the code & code. Its quite obvious that what you desire currently not working. If you have the strong need, you should consider authoring a PR and use Wagon for downloading all those files.

I am quite naive in estimating the effort. This helps to start things :monkey:

G-Ork commented 1 year ago

May i prove myself wrong but there is an option called: mavenSettingsProxyId for that documented in the plugin maven site at the end under the headline Proxy Configuration. Didn't exist my time. Did you tried this?

Look at the code. It borrows the credentials from the proxy section in your settings.xml

alan-czajkowski commented 1 year ago

@G-Ork

May i proove myself wrong but there is an option for that documented in the plugin maven site at the end under the headline Proxy Configuration. Didn't exist my time. Did you tried this?

no, this is different, this is for using a HTTP proxy for the traffic: https://maven.apache.org/guides/mini/guide-proxies.html I'm not trying to go through a proxy, I'm trying to go directly to Nexus (without proxy) to use raw-proxy repos inside of Nexus to re-point these URLs:

<cveUrlModified>https://internal-hostname/repository/proxy-raw-nist-nvd/feeds/json/cve/1.1/nvdcve-1.1-modified.json.gz</cveUrlModified>
<cveUrlBase>https://internal-hostname/repository/proxy-raw-nist-nvd/feeds/json/cve/1.1/nvdcve-1.1-%d.json.gz</cveUrlBase>
<knownExploitedUrl>https://internal-hostname/repository/proxy-raw-cisa/sites/default/files/feeds/known_exploited_vulnerabilities.json</knownExploitedUrl>

any connection to my Nexus requires credentials, and I need to reference those credentials using something like:

<serverId>internal-hostname</serverId>
<cveServerId>internal-hostname</cveServerId>

but this does not work in the plugin

alan-czajkowski commented 9 months ago

@jeremylong this seems like an important feature, is anybody looking in to this?

G-Ork commented 9 months ago

@alan-czajkowski I used a separate raw repo in nexus without credentials. I could explain my cto that the security risk is not that great with a separate repo as you open just the NIST URL. If you have strong security considerations you should have a firewall and ssl aware antivirus between your maven proxy and the Internet.

Maven central isnt a save harbour to rely on blind.

Althoug i would also like to see this plugin fixed to be in line with maven proxy feature. But in the end it is up too you to choose one of the ways mentioned including fixing and creating an PR.

jeremylong commented 9 months ago

we accept PRs...