whitewidow.rb:208:in `[]': invalid byte sequence in UTF-8 (ArgumentError)

[airatkazan]

ruby version 2.2.0

ruby whitewidow.rb -f /home/user/hack/burp/whitewidow/tmp/SQL_sites_to_check.txt

( W ( h ( i ( t ( e ( w ( i ( d ( o ( w ) ( S ( Q ( L ) ( V ( u ( l ( n ) ( S ( c ( a ( n ( n ( e ( r ) {"}/ //^\ ....... ....... ....... [19:56:30 INFO]Parsing page for SQL syntax error: http://www.lancier-cable.com/inc/print.php?HostPrint=www.lancier-cable.com&path=/&layoutpath=/doc/&title=Videos&URL=http://www.lancier-cable.com/en/en/en/en/en/create.php?cat=7&lng=en' whitewidow.rb:208:in []': invalid byte sequence in UTF-8 (ArgumentError) from whitewidow.rb:208:inblock (2 levels) in vulnerability_check' from /usr/lib/ruby/2.2.0/timeout.rb:88:in block in timeout' from /usr/lib/ruby/2.2.0/timeout.rb:32:inblock in catch' from /usr/lib/ruby/2.2.0/timeout.rb:32:in catch' from /usr/lib/ruby/2.2.0/timeout.rb:32:incatch' from /usr/lib/ruby/2.2.0/timeout.rb:103:in timeout' from whitewidow.rb:206:inblock in vulnerability_check' from whitewidow.rb:203:in each_line' from whitewidow.rb:203:invulnerability_check' from whitewidow.rb:262:in `

'

[Ekultek]

That's interesting, how many times has this happened to you?

[Ekultek]

Whitewidow already has the full path of your computer using Dir.pwd from here all you have to do is call the paths within whitewidow itself, so for you:

ruby whitewidow.rb -f tmp/SQL_sites_to_check.txt

You also don't need to use the first slash, it's already provided to make it easier.

Try dropping the full path and just using the path within whitewidow itself without the beginning slash, and let me know what happens with it, thank you for the bug

[airatkazan]

ruby whitewidow.rb -f tmp/SQL_sites_to_check.txt not work. Error whitewidow.rb:208:in []': invalid byte sequence in UTF-8 (ArgumentError) from whitewidow.rb:208:inblock (2 levels) in vulnerability_check' ...etc...

I found: $locale LANG=ru_RU.UTF-8 if i enter: $export LANG=C whitewidow work ok

[Ekultek]

Hmm, I'll look into getting it fixed permanently, if you know how please contribute

Sent from my iPhone

On Apr 16, 2016, at 4:52 PM, airatkazan notifications@github.com wrote:

$locale LANG=ru_RU.UTF-8 if i enter: $export LANG=C whitewidow work ok

— You are receiving this because you commented. Reply to this email directly or view it on GitHub

[Ekultek]

Do me a favor and post the URL you're trying to run, I have an idea of how I can permantley fix this, it has to do with the encoding error I've been getting lately and I think that trying to enforce the coding to UTF-8 will fix this

[airatkazan]

I run with -d options and have error from http://www.lancier-cable.com/inc/print.php?HostPrint=www.lancier-cable.com&path=/&layoutpath=/doc/&title=Videos&URL=http://www.lancier-cable.com/en/en/en/en/en/create.php?cat=7&lng=en'

p.s.: hmm.... now i whitedow with -d options and all be Ok. URL not http://www.lancier-cable.com...

[Ekultek]

I know what the problem is, that site, if you go to it, is a print site. It apparently holds all of that companies print jobs.. Weird

[airatkazan]

New URL with this error: [07:12:14 WARNING]URL: http://www.plantproducts.com/ca/viewproduct.php?pid=2245-- is not vulnerable, dumped to non_exploitable.txt [07:12:15 INFO]Parsing page for SQL syntax error: http://www.plantproducts.com/ca/viewproduct.php?pid=2245; [07:12:15 WARNING]URL: http://www.plantproducts.com/ca/viewproduct.php?pid=2245; is not vulnerable, dumped to non_exploitable.txt [07:12:16 INFO]Parsing page for SQL syntax error: http://www.mark-v-online.com/viewproduct.php?product_id=30' whitewidow.rb:208:in []': invalid byte sequence in UTF-8 (ArgumentError) from whitewidow.rb:208:inblock (2 levels) in vulnerability_check' from /usr/lib/ruby/2.2.0/timeout.rb:88:in block in timeout' from /usr/lib/ruby/2.2.0/timeout.rb:32:inblock in catch' from /usr/lib/ruby/2.2.0/timeout.rb:32:in catch' from /usr/lib/ruby/2.2.0/timeout.rb:32:incatch' from /usr/lib/ruby/2.2.0/timeout.rb:103:in timeout' from whitewidow.rb:206:inblock in vulnerability_check' from whitewidow.rb:203:in each_line' from whitewidow.rb:203:invulnerability_check' from whitewidow.rb:241:in `

'

[airatkazan]

[07:18:14 INFO]Parsing page for SQL syntax error: http://apologia.pvpve.net/plugins/info/info.php?page=9' whitewidow.rb:208:in []': invalid byte sequence in UTF-8 (ArgumentError) from whitewidow.rb:208:inblock (2 levels) in vulnerability_check' from /usr/lib/ruby/2.2.0/timeout.rb:88:in block in timeout' from /usr/lib/ruby/2.2.0/timeout.rb:32:inblock in catch' from /usr/lib/ruby/2.2.0/timeout.rb:32:in catch' from /usr/lib/ruby/2.2.0/timeout.rb:32:incatch' from /usr/lib/ruby/2.2.0/timeout.rb:103:in timeout' from whitewidow.rb:206:inblock in vulnerability_check' from whitewidow.rb:203:in each_line' from whitewidow.rb:203:invulnerability_check' from whitewidow.rb:241:in `

'

[airatkazan]

$cat test http://apologia.pvpve.net/plugins/info/info.php?page=9 http://www.mark-v-online.com/viewproduct.php?product_id=30 http://www.lancier-cable.com/inc/print.php?HostPrint=www.lancier-cable.com&path=/&layoutpath=/doc/&title=Videos&URL=http://www.lancier-cable.com/en/en/en/en/en/create.php?cat=7&lng=en

(this is URLs with "whitewidow.rb:208:in []': invalid byte sequence in UTF-8 (ArgumentError) from whitewidow.rb:208:inblock (2 levels) in vulnerability_check'" error)

$export LANG=C $ruby whitewidow.rb -f test

[07:25:00 INFO]Formatting file [07:25:00 INFO]Writing to temporary file.. [07:25:00 INFO]File: test, has been formatted and saved as #sites.txt in the tmp directory. [07:25:00 INFO]Let's check out this file real quick like.. [07:25:00 INFO]Parsing page for SQL syntax error: http://apologia.pvpve.net/plugins/info/info.php?page=9 [07:25:01 WARNING]URL: http://apologia.pvpve.net/plugins/info/info.php?page=9 is not vulnerable, dumped to non_exploitable.txt [07:25:02 INFO]Parsing page for SQL syntax error: http://apologia.pvpve.net/plugins/info/info.php?page=9 [07:25:04 WARNING]URL: http://apologia.pvpve.net/plugins/info/info.php?page=9 is not vulnerable, dumped to non_exploitable.txt [07:25:05 INFO]Parsing page for SQL syntax error: http://www.mark-v-online.com/viewproduct.php?product_id=30 [07:25:07 WARNING]URL: http://www.mark-v-online.com/viewproduct.php?product_id=30 is not vulnerable, dumped to non_exploitable.txt [07:25:08 INFO]Parsing page for SQL syntax error: http://apologia.pvpve.net/plugins/info/info.php?page=9 [07:25:09 WARNING]URL: http://apologia.pvpve.net/plugins/info/info.php?page=9 is not vulnerable, dumped to non_exploitable.txt [07:25:10 INFO]Parsing page for SQL syntax error: http://www.mark-v-online.com/viewproduct.php?product_id=30 [07:25:11 WARNING]URL: http://www.mark-v-online.com/viewproduct.php?product_id=30 is not vulnerable, dumped to non_exploitable.txt [07:25:12 INFO]Parsing page for SQL syntax error: http://www.lancier-cable.com/inc/print.php?HostPrint=www.lancier-cable.com&path=/&layoutpath=/doc/&title=Videos&URL=http://www.lancier-cable.com/en/en/en/en/en/create.php?cat=7&lng=en [07:25:14 WARNING]URL: http://www.lancier-cable.com/inc/print.php?HostPrint=www.lancier-cable.com&path=/&layoutpath=/doc/&title=Videos&URL=http://www.lancier-cable.com/en/en/en/en/en/create.php?cat=7&lng=en is not vulnerable, dumped to non_exploitable.txt [07:25:15 INFO]I'm truncating SQL_sites_to_check file back to 0 [07:25:15 INFO]I've run all my tests and queries, and logged all important information into /home/airat/hack/burp/whitewidow/log/SQL_VULN.LOG

All OK. Problem in encoding?

[Ekultek]

Yes I need to force the encoding to UTF-8 if the URL returns invalid bytes, I'll figure this out don't worry about it, for now just tell me which search queries produce the encoding error

Sent from my iPhone

On Apr 16, 2016, at 11:25 PM, airatkazan notifications@github.com wrote:

$cat test http://apologia.pvpve.net/plugins/info/info.php?page=9 http://www.mark-v-online.com/viewproduct.php?product_id=30 http://www.lancier-cable.com/inc/print.php?HostPrint=www.lancier-cable.com&path=/&layoutpath=/doc/&title=Videos&URL=http://www.lancier-cable.com/en/en/en/en/en/create.php?cat=7&lng=en $export LANG=C $ruby whitewidow.rb -f test

[07:25:00 INFO]Formatting file [07:25:00 INFO]Writing to temporary file.. [07:25:00 INFO]File: test, has been formatted and saved as #sites.txt in the tmp directory. [07:25:00 INFO]Let's check out this file real quick like.. [07:25:00 INFO]Parsing page for SQL syntax error: http://apologia.pvpve.net/plugins/info/info.php?page=9 [07:25:01 WARNING]URL: http://apologia.pvpve.net/plugins/info/info.php?page=9 is not vulnerable, dumped to non_exploitable.txt [07:25:02 INFO]Parsing page for SQL syntax error: http://apologia.pvpve.net/plugins/info/info.php?page=9 [07:25:04 WARNING]URL: http://apologia.pvpve.net/plugins/info/info.php?page=9 is not vulnerable, dumped to non_exploitable.txt [07:25:05 INFO]Parsing page for SQL syntax error: http://www.mark-v-online.com/viewproduct.php?product_id=30 [07:25:07 WARNING]URL: http://www.mark-v-online.com/viewproduct.php?product_id=30 is not vulnerable, dumped to non_exploitable.txt [07:25:08 INFO]Parsing page for SQL syntax error: http://apologia.pvpve.net/plugins/info/info.php?page=9 [07:25:09 WARNING]URL: http://apologia.pvpve.net/plugins/info/info.php?page=9 is not vulnerable, dumped to non_exploitable.txt [07:25:10 INFO]Parsing page for SQL syntax error: http://www.mark-v-online.com/viewproduct.php?product_id=30 [07:25:11 WARNING]URL: http://www.mark-v-online.com/viewproduct.php?product_id=30 is not vulnerable, dumped to non_exploitable.txt [07:25:12 INFO]Parsing page for SQL syntax error: http://www.lancier-cable.com/inc/print.php?HostPrint=www.lancier-cable.com&path=/&layoutpath=/doc/&title=Videos&URL=http://www.lancier-cable.com/en/en/en/en/en/create.php?cat=7&lng=en [07:25:14 WARNING]URL: http://www.lancier-cable.com/inc/print.php?HostPrint=www.lancier-cable.com&path=/&layoutpath=/doc/&title=Videos&URL=http://www.lancier-cable.com/en/en/en/en/en/create.php?cat=7&lng=en is not vulnerable, dumped to non_exploitable.txt [07:25:15 INFO]I'm truncating SQL_sites_to_check file back to 0 [07:25:15 INFO]I've run all my tests and queries, and logged all important information into /home/airat/hack/burp/whitewidow/log/SQL_VULN.LOG

All OK. Problem in encoding?

— You are receiving this because you commented. Reply to this email directly or view it on GitHub

[airatkazan]

inurl:info.php?menue= give error on http://apologia.pvpve.net/plugins/info/info.php?page=9' inurl:ViewProduct.php?misc= give error on http://www.mark-v-online.com/viewproduct.php?product_id=30'

[bararchy]

This happened for me too:

[17:20:59 INFO]I'm searching for possible SQL vulnerable sites, using search query inurl:affiliate.php?ID=

[17:21:00 SUCCESS]Site found: http://www.gravion.net/affiliate.php?id=28
[17:21:01 SUCCESS]Site found: https://gamingmouse.com/affiliate.php?id=13
[17:21:02 SUCCESS]Site found: http://www.jibberjobber.com/affiliate.php?affiliate=8
[17:21:03 SUCCESS]Site found: http://www.innpower.ca/affiliate.php?id=4
[17:21:04 SUCCESS]Site found: https://sourceforge.net/directory/?q=affiliate+php
[17:21:05 SUCCESS]Site found: http://www.salsajeans.com/affiliate.php?id=45&us=185442&tp=2
[17:21:06 SUCCESS]Site found: http://www.americanmedical-id.com/about_us/affiliate.php
[17:21:07 SUCCESS]Site found: http://home.kalatublog.com/affiliate.php
[17:21:08 SUCCESS]Site found: https://xtremestream.org/affiliate.php
[17:21:09 SUCCESS]Site found: /settings/ads/preferences?hl=iw
[17:21:10 SUCCESS]Site found: http://www.finextra.com/news/announcement.aspx?pressreleaseid=60960
[17:21:11 INFO]I've dumped possible vulnerable sites into whitewidow/tmp/SQL_sites_to_check.txt
[17:21:11 INFO]I'll run in default mode!
[17:21:11 INFO]Parsing page for SQL syntax error: http://www.gravion.net/affiliate.php?id=28'
whitewidow.rb:208:in `[]': invalid byte sequence in UTF-8 (ArgumentError)
    from whitewidow.rb:208:in `block (2 levels) in vulnerability_check'
    from /usr/lib/ruby/2.3.0/timeout.rb:91:in `block in timeout'
    from /usr/lib/ruby/2.3.0/timeout.rb:33:in `block in catch'
    from /usr/lib/ruby/2.3.0/timeout.rb:33:in `catch'
    from /usr/lib/ruby/2.3.0/timeout.rb:33:in `catch'
    from /usr/lib/ruby/2.3.0/timeout.rb:106:in `timeout'
    from whitewidow.rb:206:in `block in vulnerability_check'
    from whitewidow.rb:203:in `each_line'
    from whitewidow.rb:203:in `vulnerability_check'
    from whitewidow.rb:241:in `<main>'

[bararchy]

@Ekultek You can use this in Ruby:

# I'll Assume url is the string
url.encode(Encoding.find('UTF-8'), {invalid: :replace, undef: :replace, replace: ''}) # Force encoding to UTF8

[Ekultek]

@bararchy

If you know how to do it, why haven't you contributed to it..?

[bararchy]

@Ekultek Sorry, I just knew this one from the top of my head as my day job is working with HTTP\URLS and network programming using Ruby.

I'll really try to find time this week to go over things and see where I can help

[Ekultek]

@bararchy give me your job!

It's all good, I was just curious didn't mean to come across forceful

[Ekultek]

However I do have a question, forcing the encoding could potentially cause an issue, couldn't it? If the encoding, for lack of a better words, "doesn't want to be forced" then there's a perfectly viable URL, that could possibly be vulnerable, that's now wasted..? Isn't there a way around encoding forcing, like for example with the IO encoding errors, if I can figure that out, I'm sure it could potentially help with this problem

[bararchy]

Well, the issue here is that the HttpClient you are using can't handle non UTF8 it seems, also, most of the times a non UTF8 url's are not RFC viable.

I would enforce the encoding and just have that as a default, if someone want they can pass --no-force-encode or something of the like to run without it

[Ekultek]

That seems like it could work.. I just don't want to screw myself or others out of possible urls is all

Sent from my iPhone

On Apr 17, 2016, at 9:39 AM, Bar Hofesh notifications@github.com wrote:

Well, the issue here is that the HttpClient you are using can't handle non UTF8 it seems, also, most of the times a non UTF8 url's are not RFC viable.

I would enforce the encoding and just have that as a default, if someone want they can pass --no-force-encode or something of the like to run without it

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

[airatkazan]

guys, sorry, i can help you only as tester - i not programmer p.s.: sorry for my English )

[Ekultek]

Oh you're good man, I'll be adding you to the contributors list. I'm also working on this issue right now, I think it's fixed

Sent from my iPhone

On Apr 17, 2016, at 3:52 PM, airatkazan notifications@github.com wrote:

guys, sorry, i can help you only as tester - i not programmer p.s.: sorry for my English )

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

[airatkazan]

Ekultek, in future, i can give you ideas for your product. I know analogues, i know this market (ITsec).

[Ekultek]

Sounds like a plan to me.

Sent from my iPhone

On Apr 17, 2016, at 4:01 PM, airatkazan notifications@github.com wrote:

Ekultek, in future, i can give you ideas for your product. I know analogues, i know this market (ITsec).

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub

[airatkazan]

I'm serious. If you're interested

[Ekultek]

So I think I have this fixed, then again I added about 5000 more Google dorks.. So We'll see what happens. I'm going to release a patch for 1.0.6 within the next couple hours

[airatkazan]

hmmm... dorks it's good...) , but see (as example) functional https://github.com/googleinurl/SCANNER-INURLBR

[airatkazan]

dorks live and death in short time. People need functionality and flexibility

[Ekultek]

I'm pretty sure this is fixed. If it keeps happening, I need the search query, the url, and the error.