Closed ghost closed 2 years ago
Hello @DarkRed777 ! How are you doing?
I always though that a debugging view would be helpful for development more than a "Feature".
What fields or elements do you whink are important for a verbose output?
Hello @DarkRed777 ! How are you doing?
I always though that a debugging view would be helpful for development more than a "Feature".
What fields or elements do you whink are important for a verbose output?
Hello Dear, Maurosoria ! Thank you very much for your reply! <3 Well, I agree with you regarding debugging purposes but it's also usefull while real pentests. IMHO adjustable verbosity gives a clear image of what's going on; returned status codes, current URL, response size should be enough i think. I have lots of cases when i needed to see status code and response size for each request during forced browsing. To be honest since dirsearch has no --verbose option the workaround for me is only to looking n(tail -f) at squid logs.
1619423367.160 2 127.0.0.1 TCP_MISS/301 469 GET http://somedomain.tld/cde.txt - HIER_DIRECT/XX.YYY.ZZ.PP text/html
1619423367.174 2 127.0.0.1 TCP_MISS/301 470 GET http://somedomain.tld/cde.html - HIER_DIRECT/XX.YYY.ZZ.PP text/html
1619423367.177 2 127.0.0.1 TCP_MISS/301 468 GET http://somedomain.tld/cde.js - HIER_DIRECT/XX.YYY.ZZ.PP text/html
1619423367.182 2 127.0.0.1 TCP_MISS/301 469 GET http://somedomain.tld/cde.zip - HIER_DIRECT/XX.YYY.ZZ.PP text/html
1619423367.189 2 127.0.0.1 TCP_MISS/301 469 GET http://somedomain.tld/cde.log - HIER_DIRECT/XX.YYY.ZZ.PP text/html
1619423367.195 2 127.0.0.1 TCP_MISS/301 469 GET http://somedomain.tld/cde.rar - HIER_DIRECT/XX.YYY.ZZ.PP text/html
1619423367.200 2 127.0.0.1 TCP_MISS/301 465 GET http://somedomain.tld/cde - HIER_DIRECT/XX.YYY.ZZ.PP text/html
1619423367.207 2 127.0.0.1 TCP_MISS/301 466 GET http://somedomain.tld/cde/ - HIER_DIRECT/XX.YYY.ZZ.PP text/html
1619423367.208 2 127.0.0.1 TCP_MISS/301 475 GET http://somedomain.tld/emailPage.php - HIER_DIRECT/XX.YYY.ZZ.PP text/html
1619423367.219 2 127.0.0.1 TCP_MISS/301 475 GET http://somedomain.tld/emailPage.txt - HIER_DIRECT/XX.YYY.ZZ.PP text/html
Anyways, Thank you very much Guys for such a great tool!
Cheers,
Right now, dirsearch is printing status code, human-readable size, URL and redirect. What do you expect?
Right now, dirsearch is printing status code, human-readable size, URL and redirect. What do you expect more?
Hello @shelld3v Well, i can't see them? (There is no response size and status code in this stage) (v (latest): v0.4.1) Please note that: This is from ongoing forced browsing:: that's why i'm asking such a usefull feature --verbose flag.
1) Screenshot: https://i.ibb.co/n31nKBp/1.png (https://ibb.co/CJ9MS6L)
Thanks in advance!
- Screenshot: https://i.ibb.co/n31nKBp/1.png (https://ibb.co/CJ9MS6L)
LOL, probably this is the first time you use dirsearch. The reason that you haven't seen anything is that dirsearch hasn't found any interesting endpoints yet. Only the results are printed with status codes and sizes
Welcome to dirsearch!
- Screenshot: https://i.ibb.co/n31nKBp/1.png (https://ibb.co/CJ9MS6L)
LOL, probably this is the first time you use dirsearch. The reason that you haven't seen anything is that dirsearch hasn't found any interesting endpoints yet. Only the results are printed with status codes and sizes
Welcome to dirsearch!
LMAO) Dear @shelld3v! dirsearch is mydaily tool) It seems you don't understand me why am i asking --verbose flag. I need --verbose flag only for ongoing forced browsing to print returned status codes, current URL, response size for each ongoing request (it doesn't matter dirsearch found || not found anything). Got it?
ongoing forced browsing
You meant the ... - Last request to: ...
?
ongoing forced browsing
You meant the
... - Last request to: ...
?
Yes! To make a clear please take a look to the following output: (this is from wfuzz) That's why i'm looking --verbose flag) Hope that makes sense.
$ python wfuzz.py --filter="(c!=404)" -c -z file,wordlist/general/megabeast.txt http://192.168.0.102/FUZZ
Target: http://192.168.0.102/FUZZ Total requests: 45463
==================================================================
ID Response Lines Word Chars Request
==================================================================```
00009: C=301 7 L 20 W 235 Ch "admin" 11359: C=301 7 L 20 W 234 Ch "data" 21263: C=301 7 L 20 W 238 Ch "includes" 41255: C=301 7 L 20 W 234 Ch "test"
Total time: 36.16794 Processed Requests: 45463 Filtered Requests: 45459 Requests/sec.: 1256.997
ongoing forced browsing
You meant the
... - Last request to: ...
?Yes! To make a clear please take a look to the following output: (this is from wfuzz) That's why i'm looking --verbose flag) Hope that makes sense.
$ python wfuzz.py --filter="(c!=404)" -c -z file,wordlist/general/megabeast.txt http://192.168.0.102/FUZZ
- Wfuzz 2.1.5 - The Web Bruteforcer *
Target: http://192.168.0.102/FUZZ Total requests: 45463
================================================================== ID Response Lines Word Chars Request ==================================================================```
00009: C=301 7 L 20 W 235 Ch "admin" 11359: C=301 7 L 20 W 234 Ch "data" 21263: C=301 7 L 20 W 238 Ch "includes" 41255: C=301 7 L 20 W 234 Ch "test"
Total time: 36.16794 Processed Requests: 45463 Filtered Requests: 45459 Requests/sec.: 1256.997
There are 3 reasons I can't agree with you:
The output now is pretty good, right?
_|. _ _ _ _ _ _|_ v0.4.1
(_||| _) (/_(_|| (_| )
Extensions: php | HTTP method: GET | Threads: 30 | Wordlist size: 8914
Error Log: /home/admin/dirsearch/logs/errors-21-04-26_16-43-38.log
Target: https://[redacted]
Output File: /home/admin/dirsearch/reports/[redacted]/_21-04-26_16-43-38.txt
[16:43:38] Starting:
[16:43:40] 500 - 3KB - /.jsp
[16:43:42] 500 - 3KB - /HelloHTML.jsp
CTRL+C detected: Pausing threads, please wait...
[e]xit / [c]ontinue: e
Canceled by the user
ongoing forced browsing
You meant the
... - Last request to: ...
?Yes! To make a clear please take a look to the following output: (this is from wfuzz) That's why i'm looking --verbose flag) Hope that makes sense. $ python wfuzz.py --filter="(c!=404)" -c -z file,wordlist/general/megabeast.txt http://192.168.0.102/FUZZ
- Wfuzz 2.1.5 - The Web Bruteforcer *
Target: http://192.168.0.102/FUZZ Total requests: 45463
ID Response Lines Word Chars Request ==================================================================``` 00009: C=301 7 L 20 W 235 Ch "admin" 11359: C=301 7 L 20 W 234 Ch "data" 21263: C=301 7 L 20 W 238 Ch "includes" 41255: C=301 7 L 20 W 234 Ch "test" Total time: 36.16794 Processed Requests: 45463 Filtered Requests: 45459 Requests/sec.: 1256.997
There are 3 reasons I can't agree with you:
* The progress bar runs too fast, even we print in out, it's almost impossible for the users to read it * Adding such information will make the progress bar too long, and bugs will come in play * I can't find the use cases for this
The output now is pretty good, right?
_|. _ _ _ _ _ _|_ v0.4.1 (_||| _) (/_(_|| (_| ) Extensions: php | HTTP method: GET | Threads: 30 | Wordlist size: 8914 Error Log: /home/admin/dirsearch/logs/errors-21-04-26_16-43-38.log Target: https://[redacted] Output File: /home/admin/dirsearch/reports/[redacted]/_21-04-26_16-43-38.txt [16:43:38] Starting: [16:43:40] 500 - 3KB - /.jsp [16:43:42] 500 - 3KB - /HelloHTML.jsp CTRL+C detected: Pausing threads, please wait... [e]xit / [c]ontinue: e Canceled by the user
Well,
*"The progress bar runs too fast, even we print in out, it's almost impossible for the users to read it" Agree, but we can pause and scroll up. (Assuming in verbose mode they will be print-out from up to down? (without clobbering))
Adding such information will make the progress bar too long, and bugs will come in play Just printing them from up to down?
I can't find the use cases for this For initial adjustement (since we have --exclude-sizes, -minimal, --maximal etc flags). Then we can simply apply our rules. From other hand verbose mode can help us to get clear image of what's going on.
Thanks!
Agree, but we can pause and scroll up. (Assuming in verbose mode they will be print-out from up to down? (without clobbering))
How do you pause?
Just printing them from up to down?
I can't understand this. Can you give me an example of the output?
For initial adjustement (since we have --exclude-sizes, -minimal, --maximal etc flags)
The fact that dirsearch can do this for us. It performs calibration whenever we start a scan
1) How do you pause? CTRL+C ? Program may ask us: [e]xit / [c]ontinue [Change verbosity level]: 0
(If you familiar with "sqlmap" you may see it also supports changing verbosity level on-fly). So people can change verbosity level =0, 1, N... (onfly))
2) I can't understand this. Can you give me an example of the output? As an example here is default output from wfuzz. Everyting printed out to the terminal: (Similar output expected by dirsearch when --verbose flag set to something >= 1 (Just an example)) Nothing fancy here.
$ python wfuzz.py -p 192.168.31.33:3131 --filter="(c!=403)" -c -z file,wordlist/general/megabeast.txt http://somesite.tld/FUZZ
********************************************************
* Wfuzz 2.1.5 - The Web Bruteforcer *
********************************************************
Target: http://somesite.tld/FUZZ
Total requests: 45463
==================================================================
ID Response Lines Word Chars Request
==================================================================
00000: C=404 7 L 12 W 162 Ch "_vti_script"
00001: C=404 7 L 12 W 162 Ch "abdominal"
00002: C=404 7 L 12 W 162 Ch "pr0n"
00003: C=404 7 L 12 W 162 Ch "porn"
00004: C=404 7 L 12 W 162 Ch "MessagingManager"
00005: C=404 7 L 12 W 162 Ch "IISAdmin"
00006: C=404 7 L 12 W 162 Ch "maps"
00007: C=404 7 L 12 W 162 Ch "MSADC"
00008: C=404 7 L 12 W 162 Ch "IISSamples"
00009: C=404 7 L 12 W 162 Ch "PBServer"
00010: C=404 7 L 12 W 162 Ch "BizTalkServerDocs"
00011: C=404 7 L 12 W 162 Ch "_vti_bin"
00012: C=404 7 L 12 W 162 Ch "Printers"
00013: C=404 7 L 12 W 162 Ch "Ababa"
00014: C=404 7 L 12 W 162 Ch "Abby"
00015: C=404 7 L 12 W 162 Ch "PBSData"
00016: C=404 7 L 12 W 162 Ch "Rpc"
00017: C=404 7 L 12 W 162 Ch "abbreviating"
00018: C=404 7 L 12 W 162 Ch "abbreviation"
00019: C=404 7 L 12 W 162 Ch "abdomen"
00020: C=404 7 L 12 W 162 Ch "abdomens"
00021: C=404 7 L 12 W 162 Ch "admin"
00022: C=404 7 L 12 W 162 Ch "Scripts"
00023: C=404 7 L 12 W 162 Ch "Backup"
00024: C=404 7 L 12 W 162 Ch "NR"
00025: C=404 7 L 12 W 162 Ch "warez"
00026: C=404 7 L 12 W 162 Ch "_private"
00027: C=404 7 L 12 W 162 Ch "htmldocs"
00028: C=301 7 L 12 W 178 Ch "images"
00029: C=404 7 L 12 W 162 Ch "_vti_log"
00030: C=404 7 L 12 W 162 Ch "content"
00031: C=404 7 L 12 W 162 Ch "IISHelp"
00032: C=404 7 L 12 W 162 Ch "Aarhus"
00033: C=404 7 L 12 W 162 Ch "abducts"
00034: C=404 7 L 12 W 162 Ch "abductions"
....................SNIP..............
3) The fact that dirsearch can do this for us. It performs calibration whenever we start a scan True, it does and works great. But --verbose can give us more flexibility instead of relying only dirsearch's auto-callibartion. Verbose mode can help troubleshoot problems, give some clues to us, to help make some adjustements / callibrations in timely fashion, increase program's flexibility etc.
Huge Thanks!
Oh, I misunderstood you! You want to print ALL results, all paths that have been tested, not just paths that exist on website (paths that give real responses, not 404 or stuff like that). Hmmm, seems useless, because:
For initial adjustement (since we have --exclude-sizes, -minimal, --maximal etc flags)
The fact that dirsearch has already done this for us. It performs calibration whenever we start a scan
calibration = adjusting filters to clean wildcard results
Oh, I misunderstood you! You want to print ALL results, all paths that have been tested, not just paths that exist on website (paths that give real responses, not 404 or stuff like that). Hmmm, seems useless, because:
For initial adjustement (since we have --exclude-sizes, -minimal, --maximal etc flags)
The fact that dirsearch has already done this for us. It performs calibration whenever we start a scan
Yep!)
Auto-callibration may suck. (Sorry) Use case: Suppose you dirbusting real website. Dirsearch's callibration etc are automatically applied. Ok Lovely Blue team notices forced browsing to the website and denies access to website || applies some ACL (via .htaccess). Now website returns 403 status code to your every request. Your dirbusting goes .... as nothing had happened. Until now you blindly dirbusting the website. Think about what if you dirbusting with very big dictionary? You'll waste your time. That's why i think we need verbose mode. (on-fly changeable verbose - that would be fantastic solution!) Believe me as web application penetration is my daily job i have seen lot's of such scenarios.
$ python3 dirsearch.py -u "http://SOMESITE.COM/" --full-url -b -f -e php,txt,html,js,zip,log,rar -t 120 --proxy=http://127.0.0.1:3131 -w /root/tools/dirsearchnew/dirsearch/shuf1.txt
_|. _ _ _ _ _ _|_ v0.4.1
(_||| _) (/_(_|| (_| )
Extensions: php, txt, html, js, zip, log, rar | HTTP method: GET | Threads: 120 | Wordlist size: 1093395
Error Log: /root/tools/dirsearch26042021/dirsearch/logs/errors-21-04-26_19-05-44.log
Target: http://SOMESITE.COM/
Output File: /root/tools/dirsearch26042021/dirsearch/reports/SOMESITE.COM/_21-04-26_19-05-44.txt
[19:05:44] Starting:
0.40% | 263 req/s - Last request to: get_block.txt^Z
[1]+ Stopped python3 dirsearch.py -u "http://SOMESITE.COM/" --full-url -b -f -e php,txt,html,js,zip,log,rar -t 120 --proxy=http://127.0.0.1:3131 -w /root/tools/dirsearchnew/dirsearch/shuf1.txt
[root@localhost dirsearch]# tail -n 10 /var/log/squid/access.log
1619449561.510 0 127.0.0.1 TCP_MISS/403 476 GET http://SOMESITE.COM/mu-gb.txt - HIER_DIRECT/SOMESITE.COM text/html
1619449561.512 0 127.0.0.1 TCP_MISS/403 475 GET http://SOMESITE.COM/mu-gb.js - HIER_DIRECT/SOMESITE.COM text/html
1619449561.524 0 127.0.0.1 TCP_MISS/403 476 GET http://SOMESITE.COM/mu-gb.zip - HIER_DIRECT/SOMESITE.COM text/html
1619449561.525 1 127.0.0.1 TCP_MISS/403 476 GET http://SOMESITE.COM/mu-gb.log - HIER_DIRECT/SOMESITE.COM text/html
1619449561.533 1 127.0.0.1 TCP_MISS/403 476 GET http://SOMESITE.COM/mu-gb.rar - HIER_DIRECT/SOMESITE.COM text/html
1619449561.533 0 127.0.0.1 TCP_MISS/403 472 GET http://SOMESITE.COM/mu-gb - HIER_DIRECT/SOMESITE.COM text/html
1619449561.535 0 127.0.0.1 TCP_MISS/403 473 GET http://SOMESITE.COM/mu-gb/ - HIER_DIRECT/SOMESITE.COM text/html
1619449561.539 0 127.0.0.1 TCP_MISS/403 474 GET http://SOMESITE.COM/SQL.txt - HIER_DIRECT/SOMESITE.COM text/html
1619449561.541 0 127.0.0.1 TCP_MISS/403 474 GET http://SOMESITE.COM/SQL.php - HIER_DIRECT/SOMESITE.COM text/html
1619449561.543 0 127.0.0.1 TCP_MISS/403 475 GET http://SOMESITE.COM/SQL.html - HIER_DIRECT/SOMESITE.COM text/html
Cheers!
Auto-callibration may suck
I don't think so!
I don't think I'm going to implement this now, because of its rare use cases (really really rare). I will consider this in the future, but now we are working on so many enhancements that many users need.
Hi @DarkRed777, I decided to log all the requests (including uninteresting ones, as your suggestion) into a log file, in there you can find basic information of any request like status, method, length, ... This will be done after my PR gets merged, I hope you love it
Hi @DarkRed777, I decided to log all the requests (including uninteresting ones, as your suggestion) into a log file, in there you can find basic information of any request like status, method, length, ... This will be done after my PR gets merged, I hope you love it
Thank you very much and keep up bro! <3
Cheers!
Hello Folks! Lots of improvements had been made to dirsearch! Kudos for this Guys! I really enjoyed while playing with new version! (v0.4.1)
For me it seems only missing functionality is a: Verbosity. I think that would be really nice to see such output (via verbose mode enabled of course) while applying forced browsing using dirsearch: Especially status-code, response size may be usefull (for initial adjustements) Something like this:
Output: insenz.tar [Status: 200, Size: 2854] nail-care.bz2 [Status: 200, Size: 2513] nail-care.tar.gz [Status: 200, Size: 5564] mech.zip [Status: 200, Size: 3984] dwts [Status: 200, Size: 6841] .........
Thanks in advance & Keep Up!