OJ / gobuster

Directory/File, DNS and VHost busting tool written in Go
Apache License 2.0
9.58k stars 1.17k forks source link

Gobuster doesn't follow redirect #210

Closed ajdinre closed 4 years ago

ajdinre commented 4 years ago

So I'm running gobuster scan on a HackTheBox machine. The /music page contains a login with a login button redirecting to ../ona, and running gobuster doesn't find the ona folder.

Gobuster was ran with gobuster dir -r -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt -u -x php,html -t 100 -s 200,204,301,302,307,403

image

kali@kali:~/<removed>$ gobuster dir -r -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt -u <URL> -x php,html -t 100 -s 200,204,301,302,307,403 
===============================================================
Gobuster v3.0.1
by OJ Reeves (@TheColonial) & Christian Mehlmauer (@_FireFart_)
===============================================================
[+] Url:            <URL>
[+] Threads:        100
[+] Wordlist:       /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt
[+] Status codes:   200,204,301,302,307,403
[+] User Agent:     gobuster/3.0.1
[+] Extensions:     php,html
[+] Follow Redir:   true
[+] Timeout:        10s
===============================================================
2020/03/09 20:25:18 Starting gobuster
===============================================================
/music (Status: 200)
[ERROR] 2020/03/09 20:25:29 [!] Get http://<URL>/2007: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:29 [!] Get http://<URL>/17: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:29 [!] Get http://<URL>/top: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:29 [!] Get http://<URL>/index: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:29 [!] Get http://<URL>/rss: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:29 [!] Get http://<URL>/press: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:29 [!] Get http://<URL>/22: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:29 [!] Get http://<URL>/3: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:29 [!] Get http://<URL>/keygen: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:29 [!] Get http://<URL>/article: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:29 [!] Get http://<URL>/login: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:29 [!] Get http://<URL>/01: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:29 [!] Get http://<URL>/07: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:39 [!] Get http://<URL>/57: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:39 [!] Get http://<URL>/61: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:39 [!] Get http://<URL>/63: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:39 [!] Get http://<URL>/details: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:39 [!] Get http://<URL>/whatsnew: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:39 [!] Get http://<URL>/images.php: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:25:39 [!] Get http://<URL>/55.php: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:26:06 [!] Get http://<URL>/virtual.php: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:26:10 [!] Get http://<URL>/backgrounds.html: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:26:10 [!] Get http://<URL>/regulamin.php: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:26:10 [!] Get http://<URL>/363.html: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:26:13 [!] Get http://<URL>/1436.html: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:26:33 [!] Get http://<URL>/assist: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
[ERROR] 2020/03/09 20:26:34 [!] Get http://<URL>/1531.html: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
/artwork (Status: 200)
etc.....

It seems as if Gobuster didn't pick up the ../ona folder, I tried running it multiple times, I also checked the whole log, it never found or returned an error containing "/ona"

Dirbuster picked it up right away not sure what's going on here. Maybe it doesn't understand the double dots?

firefart commented 4 years ago

Gobuster does not parse the returned pages it only requests the files from your wordlists. So if your wordlists do not contain the word ona it will not be picked up.

OJ commented 4 years ago

Gobuster doesn't do anything "smart" with regards to dynamic modification of the wordlist based on the results of the brute force. It doesn't parse robots.txt automatically, it doesn't look at Location headers, it doesn't parse anchor tags, and it certainly doesn't look at action attributes in form tags. It's a very simple (and rather speedy) brute-forcing tool.

The case you're describing requires parsing of HTML content and dynamically adding this to the list of testable words. It's not a redirect (as you've implied in the title). /music returns a 200.

Dynamic parsing and wordlist modification isn't something gobuster will be doing. Once a page has been discovered, it's then up to you to go find out what that means rather than relying on the tool to do that for you.