search

yogeshojha / rengine

reNgine is an automated reconnaissance framework for web applications with a focus on highly configurable streamlined recon process via Engines, recon data correlation and organization, continuous monitoring, backed by a database, and simple yet intuitive User Interface. reNgine makes it easy for penetration testers to gather reconnaissance with minimal configuration and with the help of reNgine's correlation, it just makes recon effortless.
https://yogeshojha.github.io/rengine/
GNU General Public License v3.0
7.48k stars 1.13k forks source link

bug: Scan not starting when `subdomain is_default` is set to `False` #1056

Open psyray opened 11 months ago

psyray commented 11 months ago

Is there an existing issue for this?

Current Behavior

For an unexplained reason some subdomain does not launch any scan. When you click Initiate scan, task is sent to celery but the following log is present: 

rengine-celery-1       | initiate_subscan                   | WARNING | https://img-dev.xxx.com [200] `0B` `Apache` `159.114883ms`
rengine-celery-1       | remove_duplicate_endpoints         | INFO | Removing duplicate endpoints based on ['content_length', 'page_title']
rengine-celery-1       | run_command                        | INFO | rm /usr/src/scan_results/xxx.com_10/subscans/445/httpx_input.txt
rengine-celery-1       | run_command                        | WARNING | None
rengine-celery-1       | initiate_subscan                   | WARNING | Found subdomain root HTTP URL https://img-dev.xxx.com
rengine-celery-1       | ================================dir_file_fuzz
rengine-celery-1       | dir_file_fuzz                      | WARNING | Task dir_file_fuzz is RUNNING
rengine-celery-1       | dir_file_fuzz                      | WARNING | []
rengine-celery-1       | ================================http_crawl

Important line is this on

rengine-celery-1       | dir_file_fuzz                      | WARNING | []

No URL is present, so scan cannot start.

After further investigation, problem comes from the database. In the endpoint table, there is default endpoints, and for every subdomain without a default entry on base subdomain (https://www.xxx.com) scan will not start.

Here my sub that not scan image

The default entry is set while doing a target scan after that it is never set elsewhere. So if there is a problem while setting this value, subdomains are listed, but you can't launch any scan on it.

Workaround

If True is set to is_default value in the DB scan will work properly

Expected Behavior

Scan should start for a subdomain present in the table

Steps To Reproduce

Random, some domain works, others not. Need investigation

Environment

- reNgine: 2.0.1
- OS: Debian 12
- Python: 2.10
- Docker Engine: x
- Docker Compose: x
- Browser: FF 110

Anything else?

No response

github-actions[bot] commented 11 months ago

👋 Hi @psyray, Issues is only for reporting a bug/feature request. Please read documentation before raising an issue https://rengine.wiki For very limited support, questions, and discussions, please join reNgine Discord channel: https://discord.gg/azv6fzhNCE Please include all the requested and relevant information when opening a bug report. Improper reports will be closed without any response.