Security-Tools-Alliance / rengine-ng

reNgine-ng is an automated reconnaissance framework for web applications with a focus on highly configurable streamlined recon process via Engines, recon data correlation and organization, continuous monitoring, backed by a database, and simple yet intuitive User Interface.
GNU General Public License v3.0
70 stars 12 forks source link

bug(scan): commands output generates a lot of data in db #195

Open psyray opened 2 months ago

psyray commented 2 months ago

Current Behavior

Commands like nuclei, fuff or httpx generate a lot of output. Actually those output logs are stored in the db, it's a bad idea because it slows all transactions and generates bottleneck There are used, for example, on the dashboard page. And it could be linked with other records for displaying a table. ...

Expected Behavior

Huge output logs must be stored on disk or truncated, and we need to monitor this and propose to clean old scan

Steps To Reproduce

Launch a scan on vulnweb.com Export the startScan db

docker exec -it rengine-celery-1 bash -c 'poetry run -C $HOME python3 manage.py dumpdata startScan --indent 2 > startScan/fixtures/startScan.json'

Some fields have huge data in the output field image

Environment

- reNgine: 2.1.0
- OS: 
- Python: 
- Docker Engine: 
- Docker Compose: 
- Browser:

Anything else?

No response

Acknowledgements