KvasirSecurity / Kvasir

Kvasir: Penetration Test Data Management
Other
424 stars 86 forks source link

Memory Leak Issue #172

Open j4schur opened 9 years ago

j4schur commented 9 years ago

Just wanted to say something before I post an issue. Kvasir is an amazing product for being Open Source. Reminds me of Metasploit Pro without the 50k price tag. Good work guys.

So I've noticed using Kvasir that memory that's not needed isn't released back to the system. Kvasir will just keep hold the memory until you physically stop the service. I have 15GB of memory on my laptop and have postgresql running on a backend server. Kvasir will slowly use all of my memory on the laptop if I let it.

I know in your README file you say its for small engagements. But I wanted to see if I can import over 5,000 IP worth of Nessus vulnerability data and see how it handles. Other than the memory leak issue it can handle it.

Thanks for a great product. Would love to help out any way I can.

Jason

grutz commented 9 years ago

Thanks for the kind words! My goal isn't to remake MSF Pro but to enhance the collection and usage processes around penetration test data.

My guess is the memory issue is a combo of web2py, python and Kvasir's use of cache.ram. A quick test would be to do a global search/replace cache.ram to cache.disk. This could have other consequences so make sure you can revert easily. It shouldn't impact any stored data, just how web2py/Kvasir store cache data.

More on web2py's cache @ http://web2py.com/books/default/chapter/29/04/the-core#cache

I would expect Imports to eat a lot of cache/memory as it's probably the most active of tasks. I have been running a number of imports this morning and my python still has yet to breach 500 MB real memory size (based on OS X Activity Monitor): [image: Inline image 1] What version of python, web2py and OS are you using?

Kurt Grutzmacher -=- grutz@jingojango.net

On Fri, Feb 6, 2015 at 9:00 AM, j4schur notifications@github.com wrote:

Just wanted to say something before I post an issue. Kvasir is an amazing product for being Open Source. Reminds me of Metasploit Pro without the 50k price tag. Good work guys.

So I've noticed using Kvasir that memory that's not needed isn't released back to the system. Kvasir will just keep hold the memory until you physically stop the service. I have 15GB of memory on my laptop and have postgresql running on a backend server. Kvasir will slowly use all of my memory on the laptop if I let it.

I know in your README file you say its for small engagements. But I wanted to see if I can import over 5,000 IP worth of Nessus vulnerability data and see how it handles. Other than the memory leak issue it can handle it.

Thanks for a great product. Would love to help out any way I can.

Jason

— Reply to this email directly or view it on GitHub https://github.com/KvasirSecurity/Kvasir/issues/172.

j4schur commented 9 years ago

By the way thanks for the quick response. I'll have to try and replace cache.ram with cache.disk and see if that helps.

Python version = 2.7.5 OS version = CentOS 7 web2py version = 2.9.12

Today I'm not importing anything and python is using 21% of the system memory at the moment. I'm just browsing the data I imported yesterday and adding notes and things.

grutz commented 9 years ago

Are you sure this isn't active memory vs virtual memory usage? What are you using to track its usage?

Kurt Grutzmacher -=- grutz@jingojango.net

On Fri, Feb 6, 2015 at 10:58 AM, j4schur notifications@github.com wrote:

By the way thanks for the quick response. I'll have to try and replace cache.ram with cache.disk and see if that helps.

Python version = 2.7.5 OS version = CentOS 7 web2py version = 2.9.12

Today I'm not importing anything and python is using 21% of the system memory at the moment. I'm just browsing the data I imported yesterday and adding notes and things.

— Reply to this email directly or view it on GitHub https://github.com/KvasirSecurity/Kvasir/issues/172#issuecomment-73292452 .

j4schur commented 9 years ago

Sorry that would be total memory. I'm using htop for linux to track the usage. Here are the stats I'm seeing as of right now:

VIRT = 7373M RES = 5892M MEM% = 27.8

Command = python /opt/web2py/web2py -p 8443 -i 0.0.0.0 --minthreads=40 --password=XXXXX -X -K kvasir,kvasir,kvasir