search

stevespringett / nist-data-mirror

A simple Java command-line utility to mirror the CVE JSON data from NIST.
Apache License 2.0
206 stars 93 forks source link

Artifact downloads #7

Closed javixeneize closed 6 years ago

javixeneize commented 6 years ago

Hi

One question about mirroring. What is the final artifact from this step? Is it a h2.db file?

I would like to mirror the DB with this tool, and then, run Dependency check against it.

If the artifact that comes after this step is not a h2.db file, how do i run Dependency check against this artifact?

Thanks

stevespringett commented 6 years ago

This tool mirrors the NVD download site so that organizations can have their own internal mirror of the NVD. Refer to https://jeremylong.github.io/DependencyCheck/data/mirrornvd.html

javixeneize commented 6 years ago

Hi

Thanks for the update. I have already checked that page. My point is how this can be converted into a format that can be understood by dependency check (h2.db). Is there any tool that does it?

Thanks

On Tue, Sep 26, 2017 at 7:06 PM, Steve Springett notifications@github.com wrote:

This tool mirrors the NVD download site so that organizations can have their own internal mirror of the NVD. Refer to https://jeremylong.github.io/DependencyCheck/data/mirrornvd.html

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/stevespringett/nist-data-mirror/issues/7#issuecomment-332285278, or mute the thread https://github.com/notifications/unsubscribe-auth/AMK55sWlt6PgJmgrIHRlCXdhgBAzp9cuks5smT0kgaJpZM4PkbUi .

stevespringett commented 6 years ago

Not sure I understand the question. Dependency-Check should be setup to download from the mirror (instead of the NVD). When this is done, the h2 database will be created like normal.

javixeneize commented 6 years ago

Sorry I should have been clearer

I want to have on one side just dep check only, without downloading the database in every execution

On the other side, I want to have independently the db, and I have a job that downloads it every night.

It might sound strange but there is a reason behind that ;)

So, I want to execute this dependency check instance against the database that I am downloading every night with this mirroring and refer to it using the -d option

Here is where the question comes. Is this database downloaded via mirroring compatible with dependency check?

I was expecting to have an h2.db downloaded with this mirroring, since this is the format for the database downloaded with dependency check itself. but what I have seems to be json.gz for every year, and i dont think dependency check will accept that input for database using -d.

My question is... how can i reference from dependency check using -d to the database downloaded with the mirroring? Do i need to convert those files to a format that is valid for dependency check (h2.db)?

Thanks

El El mié, 27 sept 2017 a las 3:41, Steve Springett < notifications@github.com> escribió:

Not sure I understand the question. Dependency-Check should be setup to download from the mirror (instead of the NVD). When this is done, the h2 database will be created like normal.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/stevespringett/nist-data-mirror/issues/7#issuecomment-332392465, or mute the thread https://github.com/notifications/unsubscribe-auth/AMK55m1idrPado_JKP23SEopV2BZbb4Aks5smbXZgaJpZM4PkbUi .

stevespringett commented 6 years ago

This is a very common use case.

dependency-check --updateonly --data /my/datadir --cveUrl12Modified http://myhost/nvd-mirror/nvdcve-Modified.xml.gz --cveUrl20Modified http://myhost/nvd-mirror/nvdcve-2.0-Modified.xml.gz --cveUrl12Base http://myhost/nvd-mirror/nvdcve-%d.xml.gz --cveUrl20Base http://myhost/nvd-mirror/nvdcve-2.0-%d.xml.gz

This will download the contents from a mirror hosted on your internal network and will create (or update) the data directory.

From the command line this looks a bit ugly, but Maven, Ant, Gradle, and Jenkins plugins all support this use case.

Just replace 'http://myhost/nvd-mirror/' with the hostname and directory to where your mirror is actually located.

Most orgs run nist-data-mirror followed by the above command (or similar) on a nightly basis.

javixeneize commented 6 years ago

Aha. Looks clearer now

Why there are three different ndvcve? Do they contain the same info but in different formats?

Also, I was thinking on running just the dependency check cli with the update argument as it will just download the h2.db. Do you recommend mirroring instead of doing this?

And the last mandatory question... is there any difference between the h2.db and those xml.gz, or is it the same in a different format?

Thanks!

El El jue, 28 sept 2017 a las 5:16, Steve Springett < notifications@github.com> escribió:

This is a very common use case.

dependency-check --updateonly --data /my/datadir --cveUrl12Modified http://myhost/nvd-mirror/nvdcve-Modified.xml.gz --cveUrl20Modified http://myhost/nvd-mirror/nvdcve-2.0-Modified.xml.gz --cveUrl12Base http://myhost/nvd-mirror/nvdcve-%d.xml.gz --cveUrl20Base http://myhost/nvd-mirror/nvdcve-2.0-%d.xml.gz

This will download the contents from a mirror hosted on your internal network and will create (or update) the data directory.

From the command line this looks a bit ugly, but Maven, Ant, Gradle, and Jenkins plugins all support this use case.

Just replace 'http://myhost/nvd-mirror/' with the hostname and directory to where your mirror is actually located.

Most orgs run nist-data-mirror followed by the above command (or similar) on a nightly basis.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/stevespringett/nist-data-mirror/issues/7#issuecomment-332723255, or mute the thread https://github.com/notifications/unsubscribe-auth/AMK55gIm_A9isiL5ZgLgEayU6NcQOrWsks5smx2kgaJpZM4PkbUi .

stevespringett commented 6 years ago

Actually, there are four of them, not three. Yes, they are all different data feeds provided by the NVD. All four are necessary.

The xml.gz is the exact same thing that is available on the NVD. Once ODC downloads the data from the NVD (or a mirror of the NVD), it processes the feeds and creates a database (the h2 file).

Most orgs host an internal mirror of the NVD. They will then configure each instance of ODC to use the mirror.

This strategy allows orgs to use different versions of ODC throughout the org. Standardizing on a single h2 database version would force the org to only use a single version of ODC.

javixeneize commented 6 years ago

Perfect. Thanks!

For me is easier to download it directly in h2.db in the central repo and then configure Odc against the h2 itself

Thanks for everything! Really appreciated

El El vie, 29 sept 2017 a las 2:35, Steve Springett < notifications@github.com> escribió:

Actually, there are four of them, not three. Yes, they are all different data feeds provided by the NVD. All four are necessary.

The xml.gz is the exact same thing that is available on the NVD. Once ODC downloads the data from the NVD (or a mirror of the NVD), it processes the feeds and creates a database (the h2 file).

Most orgs host an internal mirror of the NVD. They will then configure each instance of ODC to use the mirror.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/stevespringett/nist-data-mirror/issues/7#issuecomment-333007516, or mute the thread https://github.com/notifications/unsubscribe-auth/AMK55to0L5SU_W_woL8DZmkdl0C5oPnEks5snElGgaJpZM4PkbUi .