Bug - Scanning issues after recent merge/ commits

yogeshojha / rengine

reNgine is an automated reconnaissance framework for web applications with a focus on highly configurable streamlined recon process via Engines, recon data correlation and organization, continuous monitoring, backed by a database, and simple yet intuitive User Interface. reNgine makes it easy for penetration testers to gather reconnaissance with minimal configuration and with the help of reNgine's correlation, it just makes recon effortless.

https://yogeshojha.github.io/rengine/

GNU General Public License v3.0

7.5k stars 1.14k forks source link

Bug - Scanning issues after recent merge/ commits #479

Closed alph4byt3 closed 3 years ago

alph4byt3 commented 3 years ago

Issue Description

Scans stop working after subdomain discovery but before probing subdomains for endpoints. It seems to be happening during the HTTPX part of the scan.

Here's a normal vuln scan using the default engine in a newly installed instance.

The build I was using before Thursday 26th was fine and that was the day new changes were committed, today I reinstalled rengine due to a power outage hence why I'm now here.

I have confirmed that this issue can be reproduced as described on a latest version/pull of reNgine: (yes / no)

yes

Technical details

None

github-actions[bot] commented 3 years ago

👋 Hi @alph4byt3, Issues is only for reporting a bug/feature request. Please read documentation before raising an issue https://rengine.wiki For very limited support, questions, and discussions, please join reNgine Discord channel: https://discord.gg/azv6fzhNCE Please include all the requested and relevant information when opening a bug report. Improper reports will be closed without any response.

yogeshojha commented 3 years ago

Weird! Let me try immediately on my older VPS build. But I don't think any changes has happened in scanning part, which would fail this. anyways let me check

yogeshojha commented 3 years ago

How did you update btw? I assume

make down && git pull && make build && make up

right?

yogeshojha commented 3 years ago

@alph4byt3 I can confirm everything is working as expected. My build was from Aug 16 and updated to the most recent. It is working as expected.

wapwn commented 3 years ago

@alph4byt3 @yogeshojha that's because httpx just updated, the returned json data is a bit different in the http_crawler function you just need to change the subdomain query to

subdomain = Subdomain.objects.get(scan_history=task, name=json_st['input'])

original code

subdomain = Subdomain.objects.get(scan_history=task, name=json_st['url'].split("//")[-1])

alph4byt3 commented 3 years ago

How did you update btw? I assume

make down && git pull && make build && make up

right?

Hey what's up, I didn't update it. I'm on a VM so I went back to a previous screenshot from a couple of days ago without reNgine installed and reinstalled it from scratch (git clone etc etc)

I'm quite busy at the moment so when I get free time later I'll go check on it again for any other things I can find.

I saw httpx did update something regarding json output so maybe wapwn above is correct

yogeshojha commented 3 years ago

Hi @wapwn Thank you very much for pointing this out! You're awesome 🌟

This has been fixed, tested on the most recent version of HTTPX. If this problem still persists, feel free to raise a new GitHub issue, or we can reopen this.

yogeshojha commented 3 years ago

Thanks @alph4byt3 for reporting this.