search

cve-search / vulnerability-lookup

Vulnerability-Lookup facilitates quick correlation of vulnerabilities from various sources, independent of vulnerability IDs, and streamlines the management of Coordinated Vulnerability Disclosure (CVD).
https://vulnerability.circl.lu
GNU Affero General Public License v3.0
136 stars 15 forks source link

Frontend not starting after install #28

Closed svniedner closed 7 months ago

svniedner commented 7 months ago

Hi, I managed to install vulnerability lookup in a docker container, and so far things appear to work well (I have logfile entries from the feeders). However, the web frontend does not show up, requests hang indefinitely. Is there any way to debug this type of behaviour?

At the same time, the RedHat CSAF collector still appears to be busy (for very long time, now): 2024-04-08 14:47:50,759 CSAFRedHat INFO:Still downloading CSAF data... 2024-04-08 14:48:00,764 CSAFRedHat INFO:Still downloading CSAF data...

Any advice is appreciated, as the project looks like a great contribution!

Best, Sven

adulau commented 7 months ago

Yep, the RedHat CSAF source is huge and takes hours to be inserted. Concerning the web interface, do you see the Python process running?

svniedner commented 7 months ago

Yes, plenty of them, I think the app server is also running. NO log file entries for access, though.

ps aux | grep python

root 154 0.0 0.1 50040 33016 ? S 12:55 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/csaf_redhat_importer root 1029 0.0 0.1 50876 36804 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/nvd_importer root 1030 0.0 0.0 120172 32548 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/cvelist_importer root 1031 0.0 0.1 53824 38620 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/cisa_known_exploited root 1032 0.0 0.0 120128 30660 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/gsd_importer root 1033 0.0 0.0 120792 32408 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/pysec_importer root 1034 0.0 0.0 120084 32624 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/github_importer root 1035 0.0 0.0 111912 29796 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/ossf_malicious_importer root 1036 0.0 0.2 90432 76612 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/csaf_certbund_importer root 1037 0.0 0.1 63180 50524 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/csaf_siemens_importer root 1038 0.0 1.9 669252 655340 pts/0 S 13:26 0:02 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/csaf_redhat_importer root 1039 0.0 0.1 50816 35108 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/csaf_cisa_importer root 1040 0.0 0.1 50944 36508 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/csaf_cisco_importer root 1041 0.0 0.1 51196 37020 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/csaf_sick_importer root 1042 0.0 0.1 50968 36664 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/csaf_ox_importer root 1043 0.0 0.1 50832 36464 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/csaf_nozominetworks_importer root 1044 0.0 0.0 33012 25516 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/start_website root 1045 0.0 0.0 29136 22144 pts/0 S 13:26 0:01 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/gunicorn -w 10 --graceful-timeout 2 --timeout 300 -b 0.0.0.0:10001 --log-level info web:app root 1111 0.0 0.1 56308 45072 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/gunicorn -w 10 --graceful-timeout 2 --timeout 300 -b 0.0.0.0:10001 --log-level info web:app root 1122 0.0 0.1 56304 45076 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/gunicorn -w 10 --graceful-timeout 2 --timeout 300 -b 0.0.0.0:10001 --log-level info web:app root 1123 0.0 0.1 56308 45084 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/gunicorn -w 10 --graceful-timeout 2 --timeout 300 -b 0.0.0.0:10001 --log-level info web:app root 1124 0.0 0.1 56308 45208 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/gunicorn -w 10 --graceful-timeout 2 --timeout 300 -b 0.0.0.0:10001 --log-level info web:app root 1125 0.0 0.1 56304 45080 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/gunicorn -w 10 --graceful-timeout 2 --timeout 300 -b 0.0.0.0:10001 --log-level info web:app root 1126 0.0 0.1 56308 45084 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/gunicorn -w 10 --graceful-timeout 2 --timeout 300 -b 0.0.0.0:10001 --log-level info web:app root 1150 0.0 0.1 56304 45220 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/gunicorn -w 10 --graceful-timeout 2 --timeout 300 -b 0.0.0.0:10001 --log-level info web:app root 1152 0.0 0.1 56304 45092 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/gunicorn -w 10 --graceful-timeout 2 --timeout 300 -b 0.0.0.0:10001 --log-level info web:app root 1157 0.0 0.1 56308 45096 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/gunicorn -w 10 --graceful-timeout 2 --timeout 300 -b 0.0.0.0:10001 --log-level info web:app root 1162 0.0 0.1 56308 45096 pts/0 S 13:26 0:00 /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/python /root/.cache/pypoetry/virtualenvs/vulnerabilitylookup-K_6IT42j-py3.11/bin/gunicorn -w 10 --graceful-timeout 2 --timeout 300 -b 0.0.0.0:10001 --log-level info web:app root 1379 0.0 0.0 3076 1408 pts/0 S+ 14:53 0:00 grep python

adulau commented 7 months ago

Awesome. So you should be able to browse the web interface via http://yourlocalip:10001 ?

svniedner commented 7 months ago

Unfortunately not... I only get the spinner. wget gives me 200 OK, but then no data, and hangs. And it complains about the lack of headers. PS: Funnily works inside the container now, so maybe more of a Docker problem. I will dig into it and keep you posted.

adulau commented 7 months ago

Did you specific the hostname in the configuration? You should have an interface similar to https://vulnerability.circl.lu/

svniedner commented 7 months ago

First of all, thanks for your support! Works now, I got entangled in docker port forwarding logic. Do you have interest in the Dockerfile?

Rafiot commented 7 months ago

Arriving after the battle, glad it works :)

If you're up for maintaining the dockerfile, sure, but as we're not using it, I'll not promise you it will be working on every update. If it fails on update, it will be a mistake on my side, but I really want to stress out that we're not testing it.

adulau commented 7 months ago

@svniedner Sure, that would be great. Make it as generic as possible as @Rafiot who are the best guys to maintain Docker ;-) For your information, we are running the production ones in LXD/LXC. Thanks a lot.

svniedner commented 7 months ago

Oh, interesting, unfortunately I have zero experience with LXC/LXC so for. What I did for now is write a Dockerfile automating the install process such that it comes up and "talks". Building and running the software is basically a one-liner with this file. What would be desirable in the future is to turn the thing into a "stack" with several containers. Here it would be good to understand how Redis/KVrocks are used, both during build (I saw requirements on where the source must be placed) and operations.

svniedner commented 7 months ago

I close the issue here for now, as the docker discussion is a bit off-topic for this request.