Typhoeus 0.5 released

[ethicalhack3r]

There have been a lot of changes so I think we will need to test everything before we support it.

I'll update the Gemfile so that users don't install it yet.

[ethicalhack3r]

Changelog here:

https://github.com/typhoeus/typhoeus/blob/master/CHANGELOG.md

[Xploit3r]

hey,

this is a continuation of my on going problem with making typhoeus work with wpscan.

I just ran:

sudo gem install --user-install typhoeus

I guess this will install it in the user directory.

here is the log:

WARNING: You don't have /root/.gem/ruby/1.9.2/bin in your PATH, gem executables will not run. Building native extensions. This could take a while... Successfully installed ffi-1.0.11 Successfully installed ethon-0.5.2 Successfully installed typhoeus-0.5.0 3 gems installed Installing ri documentation for ffi-1.0.11... Installing ri documentation for ethon-0.5.2... Installing ri documentation for typhoeus-0.5.0... Installing RDoc documentation for ffi-1.0.11... Installing RDoc documentation for ethon-0.5.2... Installing RDoc documentation for typhoeus-0.5.0...

Next, I tested whether typhoeus was installed or not:

root@bt:/pentest/web/wpscan# irb irb(main):001:0> require "typhoeus" => true irb(main):002:0> exit

and now I ran, wpscan.rb:

root@bt:/pentest/web/wpscan# ruby ./wpscan.rb --url http://targetsite.com/ --enumerate p

[ERROR] The option: timeout is invalid. Trace : /root/.gem/ruby/1.9.2/gems/ethon-0.5.2/lib/ethon/multi.rb:102:in block in set_attributes' /root/.gem/ruby/1.9.2/gems/ethon-0.5.2/lib/ethon/multi.rb:100:ineach_pair' /root/.gem/ruby/1.9.2/gems/ethon-0.5.2/lib/ethon/multi.rb:100:in set_attributes' /root/.gem/ruby/1.9.2/gems/ethon-0.5.2/lib/ethon/multi.rb:85:ininitialize' /root/.gem/ruby/1.9.2/gems/typhoeus-0.5.0/lib/typhoeus/hydra.rb:94:in new' /root/.gem/ruby/1.9.2/gems/typhoeus-0.5.0/lib/typhoeus/hydra.rb:94:ininitialize' /pentest/web/wpscan/lib/browser.rb:46:in new' /pentest/web/wpscan/lib/browser.rb:46:ininitialize' /pentest/web/wpscan/lib/browser.rb:60:in new' /pentest/web/wpscan/lib/browser.rb:60:ininstance' /pentest/web/wpscan/lib/wpscan/wp_target.rb:43:in initialize' ./wpscan.rb:49:innew' ./wpscan.rb:49:in `

'

It shows the new version of typhoeus in the error log, so I have made some success! :D

however, looks like the timeout option is not supported?

version of wpscan I am running is: v2.0r68027cb

[ethicalhack3r]

Thanks for letting us know!

In the mean time, could you downgrade to the previous version of Typhoeus until we migrate.

I think we might need to update the installation infor in the docs to reflect the new Gemfile installation.

[Xploit3r]

I checked it further in the code of browser.rb. Specific to the timeout option here is the code snippet:

root@bt:/pentest/web/wpscan/lib# cat browser.rb | grep -n timeout

29: :cache_timeout, 30: :request_timeout 46: @ hydra = Typhoeus::Hydra.new(:max_concurrency => @max_threads, :timeout => @request_timeout) 101: # TODO reload hydra (if the .load_config is called on a browser object, hydra will not have the new @max_threads and @request_timeout) 119: request.cache_timeout 168: # Used to enable the cache system if :cache_timeout > 0 169: unless params.has_key?(:cache_timeout) 170: params = params.merge(:cache_timeout => @cache_timeout)

The error message is specific to line number 46 where the timeout option is used.

I checked the change log of typhoeus and it does not mention anything specific to the timeout option.

PS: They updated typhoeus, just when I was facing a trouble with updating my typhoeus installation :(

sudo gem install --user-install typhoeus will install the latest version of typhoeus gem by default which is not compatible with wpscan.rb at the moment. How do I prevent that?

Please point me in the right direction to install the previous version of typhoeus 0.4.2 manually.

EDIT by @ethicalhack3r:

Removed reference to @ hydra github user.

[ethicalhack3r]

Try this:

$ sudo gem uninstall --user-install typhoeus $ sudo gem uninstall typhoeus

Then in WPScan's directory:

$ sudo gem install bundler && bundle install

[hydra]

Please stop emailing me. Do not use '@hydra' in comments please.

[Xploit3r]

oh, I did not know that if you prefix a username with the 'at' symbol, it will email the text. Thanks for editing ethicalhack3r.

I tried installing the bundle as per your command:

$ sudo gem install bundler && bundle install

root@bt:/pentest/web/wpscan/lib# sudo gem install bundler && bundle install Successfully installed bundler-1.2.1 1 gem installed Installing ri documentation for bundler-1.2.1... Installing RDoc documentation for bundler-1.2.1... Fetching gem metadata from https://rubygems.org/......... Fetching gem metadata from https://rubygems.org/.. Installing addressable (2.3.2) Installing crack (0.3.1) Installing diff-lcs (1.1.3) Using ffi (1.1.5) Using mime-types (1.19) Installing multi_json (1.3.6) Using nokogiri (1.5.5) Installing rspec-core (2.11.1) Installing rspec-expectations (2.11.3) Installing rspec-mocks (2.11.3) Installing rspec (2.11.0) Installing simplecov-html (0.7.1) Installing simplecov (0.7.1) Installing typhoeus (0.4.2) Installing webmock (1.8.11) Using bundler (1.2.1) Your bundle is complete! Use bundle show [gemname] to see where a bundled gem is installed.

It did install typhoeus per above output.

but I get this error:

root@bt:/pentest/web/wpscan# irb irb(main):001:0> require "typhoeus" LoadError: no such file to load -- typhoeus from (irb):1:in require' from (irb):1 from /usr/bin/irb:12:in

'

I think the reason is that the above bundle command is not installing this gem in users directory.

So, it is unable to load.

I can install the gem in user's directory this way:

sudo gem install --user-install typhoeus

but this will install the latest version of typhoeus.

so, next question is: how do I install typhoeus version 0.4.2 in user's directory?

by the way, the following command says that I do have typhoeus installed:

root@bt:/pentest/web/wpscan# gem list -d typhoeus

* LOCAL GEMS *

typhoeus (0.4.2) Authors: David Balatero, Paul Dix, Hans Hasselberg Rubyforge: http://rubyforge.org/projects/[none] Homepage: https://github.com/typhoeus/typhoeus Installed at: /var/lib/gems/1.9.2

Parallel HTTP library on top of libcurl multi.

however, the reason is it not working with wpscan.rb is due to some configuration error with the environment variables or the path.

As you can see in the log in my previous posts, when I attempt to install typhoeus gem in user directory, I receive the below warnings:

WARNING: You don't have /root/.gem/ruby/1.9.2/bin in your PATH, gem executables will not run.

It says, "gem executables will not run".

[ethicalhack3r]

$ sudo gem install --user-install typhoeus -v '0.4.2'

Hopefully we're getting somewhere :)

[Xploit3r]

And this worked :)

Thanks for the continued support.

[erwanlr]

The cache system has been removed with the 0.5 :'(

https://github.com/typhoeus/typhoeus/issues/230

[erwanlr]

There is also an issue with the latest version of webmock (1.9.0) :/

[firefart]

Implementation hint to be typhoeus 5 ready:

if Gem.loaded_specs["typhoeus"].version >= Gem::Version.create(0.5)
   params = {:followlocation => true, :maxredirs => 10}
else
   params = {:follow_location => true, :max_redirects => 10}
end
response = Typhoeus::Request.get(url, params)

[erwanlr]

Not enough, the cache system has been removed :/ See https://github.com/typhoeus/typhoeus/issues/230 :p

[firefart]

oh :( but is the cache system really necessary? When enumerating plugins/themes/users the url is different for each request, so there is no caching. Or am I missing a detail?

[erwanlr]

Yes it is :D

During aggressive scanning, it's disable (with the option :cache_timeout => 0)

However, it's very useful to avoid making a request twice on the homepage for example. (See http://code.google.com/p/wpscan/issues/detail?id=105)

Another example, when calling the has_readme? method, a request is done. Then if we check the version from the readme, it's taken from the cache

[firefart]

a ok thx :) What do you think about looking at these things in detail? I think the readme problem can be solved by extracting the version number immediatly after checking for the readme since the get request is already done. I think the only benefit of caching is when running wpscan without enumeration, and then there are only a few requests fewer than with caching. I think i will test this cache feature and the number of "real" requests made by wpscan

[ethicalhack3r]

Running WPScan through Burp (or other proxy) will show the requests made.

I think now that Typhoeus 5.x has had time to settle we should probably look into supporting it.

Should we drop support for Typhoeus < 5.x though?

[erwanlr]

To be honest, i was ready to look into Typhoeus 5.x in order to re-add the cache system. But i saw this : https://github.com/typhoeus/typhoeus/blob/master/lib/typhoeus/request/operations.rb#L54..L75

[firefart]

ok i did a little bit of testing: On Plugin enumeration without caching there are approx. 20 requests more than with caching (with a total of 2600 requests). The 20 requests are mostly the readme.txt fetched twice. When run without the enumeration option, there are some requests to the mainpage (from logindetection, xmlrpcdetection and so on) but these can be simplified. So i think caching can be deactivated with some modifications to the source. I modified my local source to be typhoeus 0.4.2 and 0.5 compatible. I will do some cleanup and commit the first version to my github repo so you can have a look on it.

[firefart]

looks like webmock is not typhoeus 5.0 ready :( There is a typhoeus 5.0 branch, but not merged into master. So this is currently a showstopper.

[TheRook]

Wow it looks like WPScan is dead, they can't even keep up with new library releases. Its a shame, a damn shame.

[fgeek]

It does not mean project is dead. This is not high priority issue in my opinion.

[ethicalhack3r]

@TheRook - Provocative posts are not welcome here, if you want to discuss issues that can be done without being an arse. For everyone else's benefit and not yours I will elaborate on my position with Typhoeus 5.x.

Typhoeus 5.0 was a big rewrite, including not being backwards compatible and as such there have been some issues with some of the Typhoeus releases. However, it has been a while now and Typhoeus has got over its initial teething problems when it moved to the 5.0 release.

Also as @FireFart mentioned above one of the libraries we use for testing did not support Typhoeus 5.x.

Keeping with Typhoeus 0.4.2 as we are now is not currently a major disadvantage, I don't think we've had 1 issue reported to us with regards to this (apart from installation but that's not version specific). There may be a couple of things we're missing out on in 5.x but nothing I can see as being essential. That said, supporting Typhoeus 5.x is something we will do and will keep looking at from time to time.

[erwanlr]

The support for Typhoeus is done, we just need to wait for the new version release (over the week, as mentioned there)

Furthermore, there is one notation which fails in the spec. I spent a few hours trying to find the cause, unsuccessfully :/

[ethicalhack3r]

:D awesome!

spec runs through fine for me in latest pull (v2.0r31f86b8), or am I missing something?

[erwanlr]

Yea, because the trouble line is commented ;p

[erwanlr]

Merge done.