search

stevespringett / nist-data-mirror

A simple Java command-line utility to mirror the CVE JSON data from NIST.
Apache License 2.0
206 stars 93 forks source link

Docker container doesn't mirror all files #103

Open calebwhitt opened 2 years ago

calebwhitt commented 2 years ago

I recently opened an issue for the DependencyCheck, and therefore found that I've been facing issues with the nist-data-mirror.

After following the instructions for the Docker container, when I run curl http://localhost I see the following:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>
 <head>
  <title>Index of /</title>
 </head>
 <body>
<h1>Index of /</h1>
<ul><li><a href="nvdcve-1.1-2002.meta"> nvdcve-1.1-2002.meta</a></li>
<li><a href="nvdcve-1.1-2003.meta"> nvdcve-1.1-2003.meta</a></li>
<li><a href="nvdcve-1.1-2004.meta"> nvdcve-1.1-2004.meta</a></li>
<li><a href="nvdcve-1.1-2005.meta"> nvdcve-1.1-2005.meta</a></li>
<li><a href="nvdcve-1.1-2006.meta"> nvdcve-1.1-2006.meta</a></li>
<li><a href="nvdcve-1.1-2007.meta"> nvdcve-1.1-2007.meta</a></li>
<li><a href="nvdcve-1.1-2008.meta"> nvdcve-1.1-2008.meta</a></li>
<li><a href="nvdcve-1.1-2009.meta"> nvdcve-1.1-2009.meta</a></li>
<li><a href="nvdcve-1.1-2010.meta"> nvdcve-1.1-2010.meta</a></li>
<li><a href="nvdcve-1.1-2011.meta"> nvdcve-1.1-2011.meta</a></li>
<li><a href="nvdcve-1.1-2012.meta"> nvdcve-1.1-2012.meta</a></li>
<li><a href="nvdcve-1.1-2013.meta"> nvdcve-1.1-2013.meta</a></li>
<li><a href="nvdcve-1.1-2014.meta"> nvdcve-1.1-2014.meta</a></li>
<li><a href="nvdcve-1.1-2015.meta"> nvdcve-1.1-2015.meta</a></li>
<li><a href="nvdcve-1.1-2016.meta"> nvdcve-1.1-2016.meta</a></li>
<li><a href="nvdcve-1.1-2017.json"> nvdcve-1.1-2017.json</a></li>
<li><a href="nvdcve-1.1-2017.json.gz"> nvdcve-1.1-2017.json.gz</a></li>
<li><a href="nvdcve-1.1-2017.meta"> nvdcve-1.1-2017.meta</a></li>
<li><a href="nvdcve-1.1-2018.meta"> nvdcve-1.1-2018.meta</a></li>
<li><a href="nvdcve-1.1-2019.json"> nvdcve-1.1-2019.json</a></li>
<li><a href="nvdcve-1.1-2019.json.gz"> nvdcve-1.1-2019.json.gz</a></li>
<li><a href="nvdcve-1.1-2019.meta"> nvdcve-1.1-2019.meta</a></li>
<li><a href="nvdcve-1.1-2020.json"> nvdcve-1.1-2020.json</a></li>
<li><a href="nvdcve-1.1-2020.json.gz"> nvdcve-1.1-2020.json.gz</a></li>
<li><a href="nvdcve-1.1-2020.meta"> nvdcve-1.1-2020.meta</a></li>
<li><a href="nvdcve-1.1-2021.json"> nvdcve-1.1-2021.json</a></li>
<li><a href="nvdcve-1.1-2021.json.gz"> nvdcve-1.1-2021.json.gz</a></li>
<li><a href="nvdcve-1.1-2021.meta"> nvdcve-1.1-2021.meta</a></li>
<li><a href="nvdcve-1.1-2022.json"> nvdcve-1.1-2022.json</a></li>
<li><a href="nvdcve-1.1-2022.json.gz"> nvdcve-1.1-2022.json.gz</a></li>
<li><a href="nvdcve-1.1-2022.meta"> nvdcve-1.1-2022.meta</a></li>
<li><a href="nvdcve-1.1-modified.json"> nvdcve-1.1-modified.json</a></li>
<li><a href="nvdcve-1.1-modified.json.gz"> nvdcve-1.1-modified.json.gz</a></li>
<li><a href="nvdcve-1.1-modified.meta"> nvdcve-1.1-modified.meta</a></li>
</ul>
</body></html>

However, on the Docker container itself, I see the following files:

/tmp/nvd # ls
nvdcve-1.1-2002.json         nvdcve-1.1-2009.json.gz      nvdcve-1.1-2016.meta
nvdcve-1.1-2002.json.gz      nvdcve-1.1-2009.meta         nvdcve-1.1-2017.json
nvdcve-1.1-2002.meta         nvdcve-1.1-2010.json         nvdcve-1.1-2017.json.gz
nvdcve-1.1-2003.json         nvdcve-1.1-2010.json.gz      nvdcve-1.1-2017.meta
nvdcve-1.1-2003.json.gz      nvdcve-1.1-2010.meta         nvdcve-1.1-2018.json
nvdcve-1.1-2003.meta         nvdcve-1.1-2011.json         nvdcve-1.1-2018.json.gz
nvdcve-1.1-2004.json         nvdcve-1.1-2011.json.gz      nvdcve-1.1-2018.meta
nvdcve-1.1-2004.json.gz      nvdcve-1.1-2011.meta         nvdcve-1.1-2019.json
nvdcve-1.1-2004.meta         nvdcve-1.1-2012.json         nvdcve-1.1-2019.json.gz
nvdcve-1.1-2005.json         nvdcve-1.1-2012.json.gz      nvdcve-1.1-2019.meta
nvdcve-1.1-2005.json.gz      nvdcve-1.1-2012.meta         nvdcve-1.1-2020.json
nvdcve-1.1-2005.meta         nvdcve-1.1-2013.json         nvdcve-1.1-2020.json.gz
nvdcve-1.1-2006.json         nvdcve-1.1-2013.json.gz      nvdcve-1.1-2020.meta
nvdcve-1.1-2006.json.gz      nvdcve-1.1-2013.meta         nvdcve-1.1-2021.json
nvdcve-1.1-2006.meta         nvdcve-1.1-2014.json         nvdcve-1.1-2021.json.gz
nvdcve-1.1-2007.json         nvdcve-1.1-2014.json.gz      nvdcve-1.1-2021.meta
nvdcve-1.1-2007.json.gz      nvdcve-1.1-2014.meta         nvdcve-1.1-2022.json
nvdcve-1.1-2007.meta         nvdcve-1.1-2015.json         nvdcve-1.1-2022.json.gz
nvdcve-1.1-2008.json         nvdcve-1.1-2015.json.gz      nvdcve-1.1-2022.meta
nvdcve-1.1-2008.json.gz      nvdcve-1.1-2015.meta         nvdcve-1.1-modified.json
nvdcve-1.1-2008.meta         nvdcve-1.1-2016.json         nvdcve-1.1-modified.json.gz
nvdcve-1.1-2009.json         nvdcve-1.1-2016.json.gz      nvdcve-1.1-modified.meta

Why would the container not be fully mirroring the files in /tmp/nvd? I'd expect to see all these files available when I reach localhost.

calebwhitt commented 2 years ago

I'm seeing this in /var/log/supervisord.log:

/var/log # cat supervisord.log
2022-01-04 18:20:52,685 INFO Set uid to user 0 succeeded
2022-01-04 18:20:52,688 INFO supervisord started with pid 1
2022-01-04 18:20:53,692 INFO spawned: 'crond' with pid 7
2022-01-04 18:20:53,694 INFO spawned: 'httpd' with pid 8
2022-01-04 18:20:53,697 INFO spawned: 'initialize_htdocs' with pid 9
2022-01-04 18:20:54,780 INFO success: crond entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-01-04 18:20:54,780 INFO success: httpd entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-01-04 18:20:54,781 INFO success: initialize_htdocs entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2022-01-04 18:21:07,988 INFO exited: initialize_htdocs (exit status 1; not expected)