Fixed non-running crond due to missing mirror user.

stevespringett / nist-data-mirror

A simple Java command-line utility to mirror the CVE JSON data from NIST.

Apache License 2.0

206 stars 93 forks source link

Fixed non-running crond due to missing mirror user. #43

Closed 0x7d7b closed 5 years ago

0x7d7b commented 5 years ago

For issue #40:

crond was not running due to the missing mirror user.
Since in the container there are multiple processes running (crond,httpd) switched to supervisord for better process and user management.
Better stdout/stderr redirection to console of all contained processes.
Switched back to HTTP port 80 by default instead of 8080 since the user for httpd differs now from the mirror process user.

stevespringett commented 5 years ago

thanks for the pr

httran13 commented 5 years ago

Hey guys, after this fixed, localhost is not reachable?

curl http://localhost/nvdcve-1.0-modified.json.gz curl: (52) Empty reply from server [ERROR] Unable to download meta file: http://localhost/nvdcve-1.0-modified.meta [ERROR] No documents exist

confirmed 80:8080 is forwarded

HuysMacBookIBM:nist-data-mirror huytran$ dockerps NAMES IMAGE STATUS PORTS mirror htran/nvdmirror:latest Up 2 minutes 80/tcp, 0.0.0.0:80->8080/tcp

0x7d7b commented 5 years ago

@httran13 The port inside the container is now 80 and not 8080 any more.

I did a quick test with docker run -it --name mirror -p 80:80 sspringett/nvdmirror. Then curl gave a valid response:

% curl --output test.gz http://localhost/nvdcve-1.0-modified.json.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  337k  100  337k    0     0  16.4M      0 --:--:-- --:--:-- --:--:-- 17.3M

% file test.gz
test.gz: gzip compressed data, from FAT filesystem (MS-DOS, OS/2, NT), original size modulo 2^32 6447250

0x7d7b commented 5 years ago

But there might be an other issue that currently causes exceptions while syncing the htdocs folder:

2019-10-18 19:33:12,369 INFO success: initialize_htdocs entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
java.util.zip.ZipException: Not in GZIP format
    at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:165)
    at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:79)
    at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:91)
    at us.springett.nistdatamirror.NistDataMirror.uncompress(NistDataMirror.java:225)
    at us.springett.nistdatamirror.NistDataMirror.doDownload(NistDataMirror.java:214)
    at us.springett.nistdatamirror.NistDataMirror.mirror(NistDataMirror.java:128)
    at us.springett.nistdatamirror.NistDataMirror.main(NistDataMirror.java:70)

But I think this is unrelated to my modifications.

httran13 commented 5 years ago

thanks that worked! @0x7d7b