Nexpose import Install/Update VulnData fails everytime

[lvets]

Hi,

I'm trying to import the Nexpose VulnData, but it fails everytime after approx. 10231 entries. Whenever I start a new import I see the following:

[] Populating list of Nexpose vulnerability ID summaries [] 68515 vulnerabilities to parse [*] Found 10231 vulnerabilities in the database already.

Then it starts importing all the vulnerabilities and after a while this stops...

Any idea what might be going on?

[grutz]

Given the size of the parsing I'd wager the web2py web service is timing out because the processing is taking a while. For anything that may take more than a minute (or 3) to process you should use the web2py scheulder and check the box accordingly on the import screen.

[lvets]

Are you talking about the "Run in background task:" option? That's ticked on every time.

[grutz]

Then if the web2py scheduler is running there should be output in the Tasks list. If the task ended with an error, that will be there. A successful run the full processing log is available.

If there was an error, check the errors/ folder for any files. These are python pickled output files and can be viewed in the web2py admin page. If there are any files can tar/zip them up and e-mail them to me I can take a look and see?

I'll run some tests later today with older Nexpose files that I know work to see if anything has changed inside web2py that could affect this.

[lvets]

Yes, I do get output in the Task Log:

 [*] Populating list of Nexpose vulnerability ID summaries
 [*] 68515 vulnerabilities to parse
 [*] Found 10231 vulnerabilities in the database already.
 [-] Updated linuxrpm-rhsa-2014-0920
 <more of this>
 [-] Updated freebsd-vid-ee23aa09-a175-11de-96c0-0011098ad87f
 [-] Updated windows-hotfix-ms15-084
 [-] Updated windows-hotfix-ms15-096

And then it just stops. There's no error and no output in the errors directory. That's all there is...

How does the import work? Does it get all the Nexpose data at once and then imports it or does it go 1 by 1 (gets 1 from Nexpose, updates Kvasir, gets next 1, etc)?

[grutz]

Does this happen with all Nexpose reports or just a really big one?

[lvets]

It doesn't really happen with a report. It's when I try to import all VulnData from Nexpose. I haven't imported anything else yet.

[grutz]

Aha! That feature should probably be removed as it has not been maintained and I don't have access to Nexpose at this time.

Honestly it's not a necessary function as the vulnerability details are already contained in the scan output so they're added to the database during the import process.