3rd party app

[ManuelFFF]

Hi,

Have you ever heard about "Vulnwhisperer" before? This is a great tool that allows to import reports data from OpenVas (and many others vulnerability scanners) into ELK stack. I tried to follow the steps described in the official documentation, but at the end I failed to have Vulnwhisperer working.

I'm just telling you this, in case you already know this application and have tried to integrate it into SELKS in the past. This would expand SELKS horizons.

If it is the case that you already know about this application and can help me with the installation, I would really appreciate it.

Links of interest: http://www.vulnwhisperer.com/#run https://github.com/HASecuritySolutions/VulnWhisperer

Thank you

[pevma]

Thank you for the feedback - much appreciated as usual !

I have not tried that application before but it looks promising. What is the err/reason that you could not install it ?

[ManuelFFF]

Hi,

The installation steps are incomplete or lacks details. I found several issues during the installation, but after some troubleshooting I was able to resolve most of them. Finally I got some Python errors and I was unable to move forward. I am stuck in the last step: connect to OpenVas. I will share my steps and also the console output errors. By the way, thank you for your interest.

Questions:

1. When cloning from GitHub, files are downloaded into "/home/user" by default. Is there any other specific location I should use instead? I don't think I should have running any application from "/home".
2. When I have to edit some configuration files (per the official steps), do I have to modify files within the source folders or somewhere in "/share" or "/opt", where some files where copied?

Steps I followed:

1. Download source files

$ git clone https://github.com/HASecuritySolutions/VulnWhisperer.git

2. Install requirements

$ sudo apt-get install zlib1g-dev libxml2-dev libxslt1-dev
$ cd VulnWhisperer
$ pip install -r requirements.txt
$ python setup.py install

3. Fill out the section you want to process in frameworks_example.ini file *Note: Open config file and fill out the section corresponding to vulnerability software you are using

$ sudo nano /home/user1/VulnWhisperer/configs/frameworks_example.ini

[openvas]
enabled = true
hostname = openvas_server_IP
port = 443
username = vulnwhisperer
password = **********
write_path=/opt/vulnwhisperer/data/openvas/
db_path=/opt/vulnwhisperer/data/database
verbose=true

4. NOT USED (Optional) [JIRA] If using Jira, fill Jira config in the config file mentioned above.
5. [ELK] Modify the IP settings in the Logstash files to accommodate your environment *Note: Need to add a Logstash config file to process logs from a new source (input/output sections)

$ sudo nano /home/user1/VulnWhisperer/resources/elk6/pipeline/3000_openvas.conf

*Note: Find ELK host section at the EOF and enter “localhost” (because ELK stack and VulnWhisperer will be running on the same server)

5.1. Copy/Move the Logstash .conf files from /VulnWhisperer/logstash/ to /etc/logstash/conf.d/

$ sudo cp /home/user1/VulnWhisperer/resources/elk6/pipeline/3000_openvas.conf /etc/logstash/conf.d/

5.2. Validate the Logstash.conf files input contains the correct location of VulnWhisper Scans in the input.file.path directory identified below:

input {
  file {
    path => "/opt/VulnWhisperer/data/openvas/*.json"
    type => json
    codec => json
    start_position => "beginning"
    tags => [ "openvas_scan", "openvas" ]
    mode => "read"
    start_position => "beginning"
    file_completed_action => "delete"

  }
}

output {
  if "openvas" in [tags] {
    stdout {
      codec => dots
    }
    elasticsearch {
      hosts => [ "localhost:9200" ]
      index => "logstash-vulnwhisperer-%{+YYYY.MM}"
    }
  }
}

6. [ELK] Import the Kibana visualizations 6.1. Log into Kibana 6.2. Go to Management (left panel last option) 6.3. Go to Saved Objects (under Kibana section to the right) 6.4. Go to Import (top right) 6.5. Import file kibana.json (previously downloaded from Github or ELK server)

7. Run Vulnwhisperer

$ vuln_whisperer -c configs/frameworks_example.ini -s openvas

*Note: In the last step, if you run the command as is in the official documentation, it will fail stating it cannot find the referenced config file/folder, so you must enter the full path

$ vuln_whisperer -c /home/user1/VulnWhisperer/configs/frameworks_example.ini -s openvas

Console output errors (section 1 before troubleshooting and section 2 after troubleshooting):

user1@server1:~/VulnWhisperer$ vuln_whisperer -c configs/frameworks_example.ini -s openvas
Traceback (most recent call last):
  File "/usr/local/bin/vuln_whisperer", line 4, in <module>
    __import__('pkg_resources').run_script('VulnWhisperer==1.8', 'vuln_whisperer')
  File "/home/user1/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 661, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/home/user1/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1432, in run_script
    .format(**locals()),
pkg_resources.ResolutionError: Script 'scripts/vuln_whisperer' not found in metadata at '/home/user1/VulnWhisperer/VulnWhisperer.egg-info'
user1@server1:~/VulnWhisperer$ sudo vuln_whisperer -c configs/frameworks_example.ini -s openvas
Traceback (most recent call last):
  File "/usr/local/bin/vuln_whisperer", line 4, in <module>
    __import__('pkg_resources').run_script('VulnWhisperer==1.8', 'vuln_whisperer')
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 666, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1446, in run_script
    exec(code, namespace, namespace)
  File "/usr/local/lib/python2.7/dist-packages/VulnWhisperer-1.8-py2.7.egg/EGG-INFO/scripts/vuln_whisperer", line 6, in <module>
    from vulnwhisp.vulnwhisp import vulnWhisperer
  File "/usr/local/lib/python2.7/dist-packages/VulnWhisperer-1.8-py2.7.egg/vulnwhisp/vulnwhisp.py", line 6, in <module>
    from frameworks.nessus import NessusAPI
  File "/usr/local/lib/python2.7/dist-packages/VulnWhisperer-1.8-py2.7.egg/vulnwhisp/frameworks/nessus.py", line 7, in <module>
    import pytz
ImportError: No module named pytz
user1@server1:~/VulnWhisperer$ cd /usr/local/lib/python2.7/dist-packages/VulnWhisperer-1.8-py2.7.egg/vulnwhisp/
user1@server1:/usr/local/lib/python2.7/dist-packages/VulnWhisperer-1.8-py2.7.egg/vulnwhisp$ sudo vuln_whisperer -c configs/frameworks_example.ini -s openvas
Traceback (most recent call last):
  File "/usr/local/bin/vuln_whisperer", line 4, in <module>
    __import__('pkg_resources').run_script('VulnWhisperer==1.8', 'vuln_whisperer')
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 666, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1446, in run_script
    exec(code, namespace, namespace)
  File "/usr/local/lib/python2.7/dist-packages/VulnWhisperer-1.8-py2.7.egg/EGG-INFO/scripts/vuln_whisperer", line 6, in <module>
    from vulnwhisp.vulnwhisp import vulnWhisperer
  File "/usr/local/lib/python2.7/dist-packages/VulnWhisperer-1.8-py2.7.egg/vulnwhisp/vulnwhisp.py", line 6, in <module>
    from frameworks.nessus import NessusAPI
  File "/usr/local/lib/python2.7/dist-packages/VulnWhisperer-1.8-py2.7.egg/vulnwhisp/frameworks/nessus.py", line 7, in <module>
    import pytz
ImportError: No module named pytz
user1@server1:/usr/local/lib/python2.7/dist-packages/VulnWhisperer-1.8-py2.7.egg/vulnwhisp$ vuln_whisperer -c configs/frameworks_example.ini -s openvas
Traceback (most recent call last):
  File "/usr/local/bin/vuln_whisperer", line 4, in <module>
    __import__('pkg_resources').run_script('VulnWhisperer==1.8', 'vuln_whisperer')
  File "/home/user1/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 661, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/home/user1/.local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1441, in run_script
    exec(code, namespace, namespace)
  File "/usr/local/lib/python2.7/dist-packages/VulnWhisperer-1.8-py2.7.egg/EGG-INFO/scripts/vuln_whisperer", line 6, in <module>
    from vulnwhisp.vulnwhisp import vulnWhisperer
ImportError: No module named vulnwhisp

After some troubleshooting I reduced the errors to only the following Python related:

ERROR:vulnWhispererOpenVAS:__init__:Unable to establish connection with OpenVAS scanner. Reason: 'NoneType' object has no attribute 'text'
ERROR:root:main:__init__() should return None, not 'bool'
ERROR: __init__() should return None, not 'bool'

[pevma]

Besides the obvious here - ImportError: No module named pytz , maybe installing the module would help? Maybe they could offer some help/guidance?

[ManuelFFF]

Hi,

The "pytz" module is being loaded when the module Nessus will be used. I checked all frameworks an classes I could and found out that in the code. But in my case, the module "nessus" is set to "false" in the config file, so in theory it should not be loaded and therefore neither the pytz module. It looks like that setting it's being ignored.

However, the "pytz" module is part of the Python libraries, of which I have versions 2.7 installed, as part of the VulnWhisperer installation and also I think Scirius use it. Performing a search I was able to confirm that the module in question is already installed.

user1@server1:~$ sudo pip install pytz
[sudo] password for user1:
Requirement already satisfied: pytz in /usr/local/lib/python2.7/dist-packages (2017.2)

/usr/local/lib/python2.7/dist-packages/pytz-2017.2.dist-info
/usr/local/lib/python2.7/dist-packages/pytz
/usr/share/python/scirius/lib/python2.7/site-packages/pytz
/usr/share/python/scirius/lib/python2.7/site-packages/pytz-2019.3.dist-info

I've submitted several requests for help to the developers of this project, but I haven't received a response in almost a week. Perhaps this is an abandoned project or they are extremely busy, although the latest publication and version of the program dates from March 2019.

Anyway, I appreciate your help and interest in this issue.

[ManuelFFF]

Hi,

Do you know any other way to export/import reports from OpenVas into ELK stack?

Thank you

[pevma]

Were you able to figure it out ?

[Yashvendra]

Hey Guys,

I ran into the same issue:

2020-07-19 15:10:30 vulnWhispererOpenVAS[81486] ERROR Unable to establish connection with OpenVAS scanner. Reason: [FAIL] Could not login to OpenVAS
2020-07-19 15:10:30 root[81486] ERROR __init__() should return None, not 'bool'

The difference is I installed through docker. So, I didn't use frameworks_example.ini, I configured Openvas settings in resources/elk6/vulnwhisperer.ini.

Then I simply ran

vuln_whisperer -c vulnwhisperer.ini -s openvas -d -v -F

It was then able to communicate to OpenVAS also in the logs showed pushing the reports to logstash.

But still when I open Kibana, it didn't show me anything. I'm stucked here now.

[pevma]

What is the Kibana err you are getting ?

[Yashvendra]

I have followed the install guide for docker-compose. I am up to the point where it is now bringing my scan data in from OPENVAS, but the problem is when I try to view the dashboard it shows no data and brings up the below error message:

Timelion: Error: in cell #1: Elasticsearch index not found: logstash-vulnwhisperer-*

There seems to be an existing issue with docker-installation as others are also facing the same problem.

[ManuelFFF]

Hi all,

I got in touch with the maintainers of this project and they state the project is like on hold for now. Not yet closed nor abandoned, but for now they will not attend any bug nor issue reports. They will continue working on this at some point in the future, but there is not ETA. I am not using Docker. VulnWhisperer is outdated and fail to interact with recent versions of GVM (aka Openvas). Perhaps updating the Python code the app would be able to work, but I am not sure how deep we need to go into the code.

@Yashvendra , for this error, you need to import all objects into KIbana first, so there is an index template named logstash-vulnwhisperer- in ELK stack. You need to go to Kibana/ Management/ Saved objects/ Import, and then upload this file: https://github.com/HASecuritySolutions/VulnWhisperer/blob/master/resources/elk6/kibana.json. Also have to add these two files to Logstash, so it can process the logs and accommodate the data into logstash-vulnwhisperer-:

• Add this to /etc/Logstash/conf.d : https://github.com/HASecuritySolutions/VulnWhisperer/blob/master/resources/elk6/pipeline/3000_openvas.conf
• Add this to /etc/Logstash : https://github.com/HASecuritySolutions/VulnWhisperer/blob/master/resources/elk6/logstash-vulnwhisperer-template.json

After adding the above two files into Logstash folders, restart Logstash process. You will see in Logstash logs that it will try (and hopefully success) add the new config and pipeline.

Good luck!