search

rapid7 / metasploit-framework

Metasploit Framework
https://www.metasploit.com/
Other
34.14k stars 13.97k forks source link

db_import: Error running db_import command #8814

Closed emilyanncr closed 1 year ago

emilyanncr commented 7 years ago

Steps to reproduce

When attempting to import a scan using db_import, I get the following error:

 db_import '/home/emily/Downloads/acunetix scans/project23 \regoth.xml' \r
[*] Importing 'Acunetix' data
[*] Import: Parsing with 'Nokogiri v1.8.0'
[*] Importing host xxx.xx.xxx.xx
[*] Importing service xxx.xx.xxx.xx:80 
[*] Importing service xxx.xx.xxx.xx:53
[*] Importing service xxx.xx.xxx.xx:53
[*] Importing service xxx.xx.xxx.xx:21
[-] Error while running command db_import: PG::UndefinedColumn: ERROR:  column web_vulns.category does not exist
LINE 1: ...lns"."name" = $5 AND "web_vulns"."query" = $6 AND "web_vulns...
                                                             ^
HINT:  Perhaps you meant to reference the column "web_vulns.category_id".
: SELECT  "web_vulns".* FROM "web_vulns" WHERE "web_vulns"."web_site_id" = $1 AND "web_vulns"."path" = $2 AND "web_vulns"."method" = $3 AND "web_vulns"."pname" = $4 AND "web_vulns"."name" = $5 AND "web_vulns"."query" = $6 AND "web_vulns"."category" = 'imported'  ORDER BY "web_vulns"."id" ASC LIMIT 1

Call stack:
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/connection_adapters/postgresql_adapter.rb:637:in `prepare'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/connection_adapters/postgresql_adapter.rb:637:in `prepare_statement'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/connection_adapters/postgresql_adapter.rb:596:in `exec_cache'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/connection_adapters/postgresql_adapter.rb:585:in `execute_and_clear'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/connection_adapters/postgresql/database_statements.rb:160:in `exec_query'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/connection_adapters/abstract/database_statements.rb:356:in `select'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/connection_adapters/abstract/database_statements.rb:32:in `select_all'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/connection_adapters/abstract/query_cache.rb:70:in `select_all'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/querying.rb:39:in `find_by_sql'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/relation.rb:639:in `exec_queries'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/relation.rb:515:in `load'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/relation.rb:243:in `to_a'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/relation/finder_methods.rb:500:in `find_nth_with_limit'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/relation/finder_methods.rb:484:in `find_nth'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/relation/finder_methods.rb:127:in `first'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/relation.rb:163:in `first_or_initialize'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/msf/core/db_manager/web.rb:353:in `block in report_web_vuln'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/msf/core/db_manager/web.rb:291:in `report_web_vuln'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/rex/parser/nokogiri_doc_mixin.rb:153:in `db_report'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/rex/parser/acunetix_nokogiri.rb:388:in `report_web_vuln'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/rex/parser/acunetix_nokogiri.rb:114:in `end_element'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/nokogiri-1.8.0/lib/nokogiri/xml/sax/document.rb:127:in `end_element_namespace'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/nokogiri-1.8.0/lib/nokogiri/xml/sax/parser.rb:110:in `parse_with'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/nokogiri-1.8.0/lib/nokogiri/xml/sax/parser.rb:110:in `parse_memory'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/nokogiri-1.8.0/lib/nokogiri/xml/sax/parser.rb:83:in `parse'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/msf/core/db_manager/import/acunetix.rb:11:in `import_acunetix_noko_stream'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/msf/core/db_manager/import/acunetix.rb:24:in `import_acunetix_xml'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/msf/core/db_manager/import.rb:95:in `import'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/msf/core/db_manager/import.rb:151:in `import_file'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/msf/ui/console/command_dispatcher/db.rb:1388:in `block (3 levels) in cmd_db_import'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/msf/ui/console/command_dispatcher/db.rb:1381:in `each'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/msf/ui/console/command_dispatcher/db.rb:1381:in `block (2 levels) in cmd_db_import'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/msf/ui/console/command_dispatcher/db.rb:1375:in `each'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/msf/ui/console/command_dispatcher/db.rb:1375:in `block in cmd_db_import'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.9/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/msf/ui/console/command_dispatcher/db.rb:1370:in `cmd_db_import'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/rex/ui/text/dispatcher_shell.rb:430:in `run_command'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/rex/ui/text/dispatcher_shell.rb:392:in `block in run_single'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/rex/ui/text/dispatcher_shell.rb:386:in `each'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/rex/ui/text/dispatcher_shell.rb:386:in `run_single'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/rex/ui/text/shell.rb:205:in `run'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/metasploit/framework/command/console.rb:48:in `start'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/lib/metasploit/framework/command/base.rb:82:in `start'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/gems/metasploit-framework-4.15.4/msfconsole:48:in `<top (required)>'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/bin/msfconsole:23:in `load'
/opt/metasploit/apps/pro/vendor/bundle/ruby/2.3.0/bin/msfconsole:23:in `<main>'

System stuff

OS: Kali Linux Version: Kali GNU/Linux Rolling \n \l msf > version\r Framework: 4.15.4-dev Console : 4.15.4-dev Installed with [ ] Commercial/Community installer (from http://www.rapid7.com/products/metasploit/download.jsp)

After the latest upgrade and dist-upgrade something funky was happening when running msfconsole. I ended up uninstalling metasploit all together so I could reinstall and hopefully fix the problem. Installing it from the command line using apt-get install, however, kept giving me an error when trying to start metasploit from the command line. I googled the error and came across an answer which messed things up a bit more. So after some fiddling and troubleshooting I tried installing it again, this time using the installer provided on rapid7. Everything was successful and I was able to start the framework from the command line without error AND without error when running msfconsole. Prior to this update, the db_import command was working just fine so something must of gone awry during this whole process. Or it might have nothing to do with it. I'm too novice to understand the log output, so any help is appreciated.

Thanks in advance!

pbarry-r7 commented 7 years ago

Hey @emilyanncr, might you be willing/able to share the XML file you're hitting the error with? If so, plz email to msfdev [at] metasploit [dot] com.

FWIW, if you find yourself in a spot in the future and want to install the latest framework release on Kali, we have some Omnibus builds available (see the section titled "Installing the Metasploit Framework on Linux"):

https://help.rapid7.com/metasploit/Content/installation-and-updates/installing-msf.html

Thx!

emilyanncr commented 7 years ago

Thanks so much! I appreciate it. I was able to upload it using the upload function in metasploit community but when running a module, I get the same errors. I will email the xml file. Thanks so much for your help!!

busterb commented 7 years ago

did you send the XML file?

bcoles commented 1 year ago

This issue appears to be specific to Metasploit Pro. The following related issues are likely duplicates:

Closing this issue as there has been no feedback for almost 6 years.

I was not able to reproduce this issue on the open source version of Metasploit.

I'm not sure what Acunetix log format was imported above, nor what format is expected. Acunetix supports multiple different report formats; however, Acunetix reports are all PDF or HTML. Acunetix also supports a few XML exports.

Attached a sample XML export from Acunetix Vulnerability Scanner version 11.0.1732711618 (trial) run against metasploitable2. I terminated the scan before completion but the XML export contains plenty of data. This matches the DBManager::Import::Acunetix import regex and was imported successfully.

https://github.com/rapid7/metasploit-framework/blob/e646c0c6e23c11983f3a0b77c4c109da71982680/lib/msf/core/db_manager/import.rb#L433-L434

It is worth noting that no vulnerabilities were imported. I've created issue #17932 to track this.

msf6 > db_import acunetix.xml 
[*] Importing 'Acunetix' data
[*] Import: Parsing with 'Nokogiri v1.14.2'
[*] Importing host 192.168.200.142
[*] Importing service 192.168.200.142:80
[*] Successfully imported /root/Desktop/metasploit-framework/acunetix.xml
msf6 > hosts

Hosts
=====

address          mac  name             os_name  os_flavor  os_sp  purpose  info  comments
-------          ---  ----             -------  ---------  -----  -------  ----  --------
192.168.200.142       192.168.200.142  Unknown                    device

msf6 > services
Services
========

host             port  proto  name  state  info
----             ----  -----  ----  -----  ----
192.168.200.142  80    tcp    http  open

msf6 > creds
Credentials
===========

host  origin  service  public  private  realm  private_type  JtR Format
----  ------  -------  ------  -------  -----  ------------  ----------

vmsf6 > vulns

Vulnerabilities
===============

Timestamp  Host  Name  References
---------  ----  ----  ----------

msf6 > notes

Notes
=====

 Time                   Host             Service  Port  Protocol  Type           Data
 ----                   ----             -------  ----  --------  ----           ----
 2023-04-25 14:35:56 U  192.168.200.142                           host.imported  {:filename=>"/root/Desktop/metasploit-framework/acunetix.xml", :type=>"Acunetix", :
 TC                                                                              time=>2023-04-25 14:35:56.345537239 UTC}

msf6 > wmap_vulns -l
msf6 > wmap_sites -l
[*] Available sites
===============

     Id  Host             Vhost            Port  Proto  # Pages  # Forms
     --  ----             -----            ----  -----  -------  -------
     0   192.168.200.142  192.168.200.142  80    http   0        0

msf6 >