search

fkie-cad / FACT_core

Firmware Analysis and Comparison Tool
https://fkie-cad.github.io/FACT_core
GNU General Public License v3.0
1.26k stars 225 forks source link

Broken cve database link breaks installation #545

Closed firmianay closed 3 years ago

firmianay commented 3 years ago

similar problem on ubuntu18.04 #435

firmy@firmy-vm:~/FACT_core$ ./src/install.py 
[2021-02-25 13:44:32][install][INFO]: FACT Installer 1.1
[2021-02-25 13:44:32][install][INFO]: Installing apt-transport-https
[2021-02-25 13:44:32][common][INFO]: Updating system
[2021-02-25 13:44:39][install][INFO]: Installing python3 python3-dev build-essential automake autoconf libtool git unzip
[2021-02-25 13:44:40][install][INFO]: Installing testresources
[2021-02-25 13:44:44][install][INFO]: Removing python3-pip python3-setuptools python3-wheel
[2021-02-25 13:44:45][common][INFO]: Installing python3 pip
[2021-02-25 13:45:38][install][INFO]: Installing setuptools==49.6.0
[2021-02-25 13:45:42][install][INFO]: Installing libmagic-dev
[2021-02-25 13:45:43][install][INFO]: Installing libfuzzy-dev
[2021-02-25 13:45:43][install][INFO]: Installing python3-tlsh
[2021-02-25 13:45:44][install][INFO]: Installing ssdeep
[2021-02-25 13:45:46][install][INFO]: Installing git+https://github.com/fkie-cad/fact_helper_file.git
[2021-02-25 13:45:54][install][INFO]: Installing psutil
[2021-02-25 13:45:56][install][INFO]: Installing pytest==6.1.2 pytest-cov pylint python-magic xmltodict yara-python==3.7.0 appdirs
[2021-02-25 13:46:31][install][INFO]: Installing lief==0.10.1
[2021-02-25 13:46:33][install][INFO]: Installing requests
[2021-02-25 13:46:35][install][INFO]: Installing pymongo pyyaml
[2021-02-25 13:46:54][install][WARNING]: Could not update python package pyyaml. Was not installed using pip originally
[2021-02-25 13:46:54][common][WARNING]: variety spec not overwritten
[2021-02-25 13:46:54][install][INFO]: Installing hurry.filesize
[2021-02-25 13:46:56][install][INFO]: Installing git+https://github.com/fkie-cad/common_helper_files.git
[2021-02-25 13:46:59][install][INFO]: Installing git+https://github.com/fkie-cad/common_helper_mongo.git
[2021-02-25 13:47:04][install][INFO]: Installing git+https://github.com/mass-project/common_helper_encoder.git
[2021-02-25 13:47:09][install][INFO]: Installing git+https://github.com/fkie-cad/common_helper_filter.git
[2021-02-25 13:47:14][install][INFO]: Installing git+https://github.com/fkie-cad/common_helper_process.git
[2021-02-25 13:47:19][install][INFO]: Installing werkzeug==0.16.1
[2021-02-25 13:47:22][install][INFO]: Installing flask flask_restful flask_security flask_sqlalchemy flask-paginate Flask-API uwsgi bcrypt python-dateutil si-prefix email-validator matplotlib
[2021-02-25 13:48:15][frontend][INFO]: Install static jstree content
[2021-02-25 13:48:18][frontend][INFO]: Install static angularJS content
[2021-02-25 13:48:18][frontend][INFO]: Install static charts.js content
[2021-02-25 13:48:18][frontend][INFO]: Install static vis-network content
[2021-02-25 13:48:18][frontend][INFO]: Installing highlight js
[2021-02-25 13:48:24][frontend][INFO]: Install static https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css content
[2021-02-25 13:48:24][frontend][INFO]: Install static https://cdnjs.cloudflare.com/ajax/libs/bootstrap-datepicker/1.8.0/css/bootstrap-datepicker.standalone.css content
[2021-02-25 13:48:24][frontend][INFO]: Install static https://cdnjs.cloudflare.com/ajax/libs/jquery/1.12.1/jquery.min.js content
[2021-02-25 13:48:24][frontend][INFO]: Install static https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.7/umd/popper.min.js content
[2021-02-25 13:48:24][frontend][INFO]: Install static https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/js/bootstrap.min.js content
[2021-02-25 13:48:24][frontend][INFO]: Install static https://cdnjs.cloudflare.com/ajax/libs/bootstrap-datepicker/1.8.0/js/bootstrap-datepicker.js content
[2021-02-25 13:48:24][frontend][INFO]: Install static https://raw.githubusercontent.com/moment/moment/develop/moment.js content
[2021-02-25 13:48:24][frontend][INFO]: Creating directory for authentication
[2021-02-25 13:48:24][frontend][INFO]: Initializing docker container for radare
[2021-02-25 13:48:28][frontend][INFO]: Pulling pdf report container
[2021-02-25 13:48:33][db][INFO]: Setting up mongo database
[2021-02-25 13:48:33][install][INFO]: Installing mongodb
[2021-02-25 13:48:34][db][INFO]: Initialize database
[2021-02-25 13:48:35][install][INFO]: Installing python-dev python-setuptools
[2021-02-25 13:48:36][install][INFO]: Installing libjpeg-dev
[2021-02-25 13:48:37][install][INFO]: Installing libssl-dev python3-tk
[2021-02-25 13:48:38][install][INFO]: Installing pluginbase Pillow cryptography pyopenssl matplotlib docker networkx
[2021-02-25 13:49:00][backend][INFO]: Installing yara
[2021-02-25 13:49:00][install][INFO]: Installing bison flex
[2021-02-25 13:49:01][backend][INFO]: skipping yara installation (already installed)
[2021-02-25 13:49:01][backend][INFO]: Building fact extraction container
[2021-02-25 13:49:09][install][INFO]: Installing git+https://github.com/fkie-cad/common_helper_yara.git
[2021-02-25 13:49:12][install][INFO]: Installing git+https://github.com/mass-project/common_analysis_base.git
[2021-02-25 13:49:18][backend][INFO]: Installing plugins
[2021-02-25 13:49:18][backend][INFO]: Running ../plugins/analysis/ip_and_uri_finder/install.sh
[2021-02-25 13:49:30][backend][INFO]: Running ../plugins/analysis/crypto_hints/install.sh
[2021-02-25 13:49:32][backend][INFO]: Running ../plugins/analysis/qemu_exec/install.sh
[2021-02-25 13:49:32][backend][INFO]: Running ../plugins/analysis/binwalk/install.sh
[2021-02-25 13:49:53][backend][INFO]: Running ../plugins/analysis/oms/install.sh
[2021-02-25 13:49:59][backend][INFO]: Running ../plugins/analysis/cve_lookup/install.sh
Traceback (most recent call last):
  File "./src/install.py", line 183, in <module>
    install()
  File "./src/install.py", line 171, in install
    backend(distribution)
  File "/home/firmy/FACT_core/src/install/backend.py", line 45, in main
    _install_plugins(distribution)
  File "/home/firmy/FACT_core/src/install/backend.py", line 109, in _install_plugins
    raise InstallationError('Error in installation of {} plugin\n{}'.format(Path(install_script).parent.name, shell_output))
helperFunctions.install.InstallationError: Error in installation of cve_lookup plugin
------------------------------------
   install cve lookup dependencies  
------------------------------------
/usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
/usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
Requirement already satisfied: pyxdameraulevenshtein in /usr/local/lib/python3.6/dist-packages (1.7.0)
Setting up database
ERROR:root:URLs are invalid. URL format might have been changed or website might have moved.
firmianay commented 3 years ago

my bad... the true issue as below:

[2021-02-25 14:30:10][backend][INFO]: Installing plugins
[2021-02-25 14:30:10][backend][INFO]: Running ../plugins/analysis/ip_and_uri_finder/install.sh
[2021-02-25 14:30:16][backend][INFO]: Running ../plugins/analysis/crypto_hints/install.sh
[2021-02-25 14:30:19][backend][INFO]: Running ../plugins/analysis/qemu_exec/install.sh
[2021-02-25 14:30:19][backend][INFO]: Running ../plugins/analysis/binwalk/install.sh
[2021-02-25 14:30:39][backend][INFO]: Running ../plugins/analysis/oms/install.sh
[2021-02-25 14:30:43][backend][INFO]: Running ../plugins/analysis/cve_lookup/install.sh
Traceback (most recent call last):
  File "./src/install.py", line 183, in <module>
    install()
  File "./src/install.py", line 171, in install
    backend(distribution)
  File "/home/firmy/FACT_core/src/install/backend.py", line 45, in main
    _install_plugins(distribution)
  File "/home/firmy/FACT_core/src/install/backend.py", line 109, in _install_plugins
    raise InstallationError('Error in installation of {} plugin\n{}'.format(Path(install_script).parent.name, shell_output))
helperFunctions.install.InstallationError: Error in installation of cve_lookup plugin
------------------------------------
   install cve lookup dependencies  
------------------------------------
/usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
/usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
Requirement already satisfied: pyxdameraulevenshtein in /usr/local/lib/python3.6/dist-packages (1.7.0)
Updating existing database
ERROR:root:CVE tables do not exist! Did you mean import CVE?
jstucke commented 3 years ago

Could you try deleting your cve database file src/plugins/analysis/cve_lookup/internal/cve_cpe.db and rerunning the installation file of the plugin src/plugins/analysis/cve_lookup/install.sh? The links seem to work for me so maybe it was just a temporary issue.

firmianay commented 3 years ago

It worked, seems a problem with my network. thanks