`nmap2db` cannot find screenshots from script `http-screenshot` in results from `runscans`

[W1nd0x]

Hey, i'm trying to import the resultsm but I get the following error:

root@30dcfd20aea8:/home# nmap2db -c ROUTABLE-CAMPAIGN-001 -s MySource -r scans/ROUTABLE/up
Traceback (most recent call last):
  File "/usr/local/bin/nmap2db", line 114, in <module>
    main()
  File "/usr/local/bin/nmap2db", line 108, in main
    merge=args.merge,
  File "/usr/local/lib/python2.7/dist-packages/ivre/db/__init__.py", line 337, in store_scan
    return store_scan_function(fname, filehash=scanid, **kargs)
  File "/usr/local/lib/python2.7/dist-packages/ivre/db/__init__.py", line 358, in store_scan_xml
    parser.parse(utils.open_file(fname))
  File "/usr/lib/python2.7/xml/sax/expatreader.py", line 107, in parse
    xmlreader.IncrementalParser.parse(self, source)
  File "/usr/lib/python2.7/xml/sax/xmlreader.py", line 123, in parse
    self.feed(buffer)
  File "/usr/lib/python2.7/xml/sax/expatreader.py", line 210, in feed
    self._parser.Parse(data, isFinal)
  File "/usr/lib/python2.7/xml/sax/expatreader.py", line 307, in end_element
    self._cont_handler.endElement(name)
  File "/usr/local/lib/python2.7/dist-packages/ivre/xmlnmap.py", line 899, in endElement
    os.path.dirname(self._fname), fname)) as fdesc:
IOError: [Errno 2] No such file or directory: u'scans/ROUTABLE/up/72/11/131/screenshot-72.11.131.101-80.jpg'

Why is this happening?

[W1nd0x]

After deleting the file cans/ROUTABLE/up/72/11/131 where the screenshot should be in, it worked.

[p-l-]

Hi,

Thanks for your report. Where was the screenshot file located?

[W1nd0x]

Hi,

Okay, somehow a few screenshots get placed in the home file where the directory scans also is in. So example: /home/screenshot.1.1.1.1.jpg - Bug? /home/scans/ROUTABLE/up/1/1/1/screenshot.1.1.1.1jpg - Where it should be placed?

When importing it goes for the ROUTABLE map but a few files get placed in the /home/ so importing fails since they are not in the correct directory somehow.

[p-l-]

The easiest way to fix that will be to change the script http-screenshot so that it outputs the absolute path of the image.

[W1nd0x]

Do you mind to do that for me and upload it somewhere so I can replace the file easly?

[p-l-]

I'll do that soon (I don't know how for now), progress will be tracked here. When the issue is fixed, the code will be merge into master (and new Docker images will be built).

Stay tuned!

[fccagou]

Hello

Some more informations

Contexte

runscans --network 1.2.3.0/24 --output=XMLFork

• This generates scans/NET-1.2.3.0_24/up/1/2/3/ tree with 5 nmap results (thats ok).
• The screenshots are located in the same dir as scans (5 screenshots)

scans/NET-1.2.3.0_24/up/1/2/3/14.xml
scans/NET-1.2.3.0_24/up/1/2/3/2.xml
scans/NET-1.2.3.0_24/up/1/2/3/25.xml
scans/NET-1.2.3.0_24/up/1/2/3/254.xml
scans/NET-1.2.3.0_24/up/1/2/3/28.xml
screenshot-1.2.3.2-80.jpg
screenshot-1.2.3.25-80.jpg
screenshot-1.2.3.254-443.jpg
screenshot-1.2.3.254-631.jpg

I try passing around

I've moved all screenshots files to the scans/NET-1.2.3.0_24/up/1/2/3/ :)

scans/NET-1.2.3.0_24/up/1/2/3/14.xml
scans/NET-1.2.3.0_24/up/1/2/3/2.xml
scans/NET-1.2.3.0_24/up/1/2/3/25.xml
scans/NET-1.2.3.0_24/up/1/2/3/254.xml
scans/NET-1.2.3.0_24/up/1/2/3/28.xml
scans/NET-1.2.3.0_24/up/1/2/3/screenshot-1.2.3.2-80.jpg
scans/NET-1.2.3.0_24/up/1/2/3/screenshot-1.2.3.25-80.jpg
scans/NET-1.2.3.0_24/up/1/2/3/screenshot-1.2.3.254-443.jpg
scans/NET-1.2.3.0_24/up/1/2/3/screenshot-1.2.3.254-631.jpg

fail

and run

  nmap2db -c TEST-NETWORK-001 -s ivreclient -r scans/NET-1.2.3.0_24/up
  Traceback (most recent call last):
    File "/usr/local/bin/nmap2db", line 114, in <module>
      main()
    File "/usr/local/bin/nmap2db", line 108, in main
      merge=args.merge,
    File "/usr/local/lib/python2.7/dist-packages/ivre/db/__init__.py", line 336, in store_scan
      raise ValueError("Unknown file type %s" % fname)
  ValueError: Unknown file type scans/NET-1.2.3.0_24/up/1/2/3/screenshot-1.2.3.25-80.jpg

• only 2 of the 5 results are imported in the db (ko)
• I can see the 2 results in ivreweb (ok)
• I can see the 2 screenshots (ok)

my 2 cts

[p-l-]

@fccagou that's "normal", since the files are located in the directory you're trying to import.

We should probably catch in nmap2db the exception we raise in ivre.db, warn the user and continue. I'll do that first step soon.

[p-l-]

I have (for now) no idea how to fix that in the NSE script.

I have implemented some workarounds to at least avoid crashes and to "work in some cases".

Can you (@fccagou et @W1nd0x) test and let me know if that works?

[p-l-]

I have just merged #167 so that you can try it. Docker images should be on their way.

[fccagou]

Tested in docker built from sources. It's OK for me @p-l-

• Screenshots are in local dir after scan
• All hosts are scanned
• all screenshots are imported in db accessed from web.