VCR allowing new requests when set to :record => :none

[pboling]

This is a VCR & Webmock problem.

With VCR version 1.10.3, Webmock version 1.6.4, and with my VCR.config like this:

c.stub_with :webmock
c.default_cassette_options = {
  :match_requests_on => [:uri, :method, :body],
  :record => :none
}

I get the expected error "Real HTTP connections are disabled..."

With VCR version 1.11.3, Webmock version 1.7.8, and with my VCR.config like this:

c.stub_with :webmock
c.default_cassette_options = {
  :match_requests_on => [:uri, :method, :body],
  :record => :none
}

I do not get the error "Real HTTP connections are disabled...", and the live requests are made.

With VCR version 2.0.0.beta2, Webmock version 1.7.8, and with my VCR.configure like this:

c.hook_into :webmock
c.default_cassette_options = {
  :match_requests_on => [:uri, :method, :body],
  :record => :none
}

I do not get the error "Real HTTP connections are disabled...", and the live requests are made.

This is certainly a bug in Webmock, but it does effectively break VCR. :(

[myronmarston]

Thanks for reporting this. However, I can't reproduce it. I created a gist with your configuration and ran it on 1.8.7 and 1.9.2 using WebMock 1.7.8 and both VCR 1.11.3 and 2.0.0.beta2, and I got the error every time. You can see my output there.

So...there must be some edge case that your code is exercising that I'm not aware of. Can you come up with a minimal example (similar to my gist) that reproduces the issue? I'm not going to be able to do anything about this until I have a reproducible example.

Thanks!

[pboling]

It is difficult to track down how this is happening, but I have made some progress. I am using ruby 1.9.2p136 (2010-12-25) [x86_64-linux] and VCR 2.0.0.beta2.

When I make HTTP requests as you have in your gist it does work. It fails to raise the "Real HTTP connections are disabled" error when I make my SOAP service requests like this (output):

! CONNECT TO server-name:32XXX ! CONNECTION ESTABLISHED POST /datasvc HTTP/1.1 Content-Type: text/xml; charset=utf-8 SOAPAction: "getUserTargetedData" User-Agent: RUBYJEDI-SOAP4R/1.5.8 (httpclient.rb/280, ruby 1.9.2 (2010-12-25) [x86_64-linux]) Date: Tue, 29 Nov 2011 19:19:59 GMT Content-Length: 662 Host: server-name:32XXX

It will require some work to distill something down to where I could get it in a repeatable gist though.

EDIT: I don't think I can distill it down enough, We are using heavily patched soap libraries for our internal service calls. These are the ones we need to record in the cassettes. VCR version 1.10.3 caught them, while 1.11.3 and 2.0.0.beta2 do not.

We have begun working on an alternate solution for mocking requests with Faker based on our WSDLs but this will be inadequate for some types of tests, so I am hoping we can get VCR working again.

[myronmarston]

Hmm....it looks like you're using soap4r which in turn uses httpclient. I've never used either, so I'm not sure how much help I can be until you come up with a repeatable example.

I think the bug is almost likely a regression in WebMock between 1.6.4 and 1.7.8. Can you use this list of commits to bisect to the commit that broke it? You could probably narrow down that list even more by just looking at the commits that changed the httpclient adapter.

Some of the HTTP libraries have many different ways to be used and it can be difficult for a library like WebMock to cover them all--so there may be a way that soap4r is using HTTP client that causes it to take a different code path and skip WebMock's logic so that VCR is never invoked.

FWIW, VCR 2.0.0.beta3 will require WebMock 1.7.8 or higher...so I'd like to get this bug fixed (wherever it is) so that it's not an issue with the final VCR 2.0 release.

[myronmarston]

EDIT: I don't think I can distill it down enough, We are using heavily patched soap libraries for our internal service calls. These are the ones we need to record in the cassettes. VCR version 1.10.3 caught them, while 1.11.3 and 2.0.0.beta2 do not.

I don't think you need to come up with a distilled example from your patched soap library. I bet SOAP4R is just using httpclient in a particular fashion that doesn't work correctly with the current WebMock. If you can dig into the soap4r source and come up with some example code that uses HTTP client in the same way, it may be sufficient. I've dealt with several other similar issues with other HTTP libraries (i.e. where on gem is using it in a particular way that isn't the "normal" way I was testing/using the HTTP library) and this approach has worked well for troubleshooting.

[pboling]

For our soap stack to work we need webmock 1.6.4 to use 'soap/streamHandler', while soap4r uses httpclient. This setup worked as long as we forcefully require 'soap/streamHandler' prior to loading webmock. My gut tells me this is somehow related. Still digging.

we have:

# This needs to be loaded before webmock is loaded, soap4r uses httpclient
require 'soap/streamHandler'

# at one point we had to configure this separately
VCR::Config.stub_with :webmock
# rest of config
VCR.config do |c|
 ...
end```

[myronmarston]

This setup worked as long as we forcefully require 'soap/streamHandler' prior to loading webmock. My gut tells me this is somehow related.

This may be related. WebMock 1.7 had some changes to deal with this.

[myronmarston]

Hey Peter,

I just released VCR 2.0.0 RC1. This is the only issue left. I'd love to address this before VCR 2.0.0 final if possible, so the issue count can be zero :). Let me know if you make any progress on coming up with an isolated example. I have access to the WebMock project as well so I can push a fix there if that is indeed where the bug is.

Thanks, Myron

[christophermanning]

I get this error even when using fakeweb. Instead, it seems to be related to net-http-persistent

I was using https://gist.github.com/908795 to test the error and when using gem install mechanize --version=2.0.1 and then disabling networking, it gives the Real HTTP connections are disabled error, but when switching to Mechanize 2.1 and disabling networking, it gives the error:

/home/c/.rvm/gems/ruby-1.9.2-p290@vcr_test/gems/net-http-persistent-2.3.3/lib/net/http/persistent/ssl_reuse.rb:29:in `initialize': getaddrinfo: Name or service not known (SocketError)
from /home/c/.rvm/gems/ruby-1.9.2-p290@vcr_test/gems/net-http-persistent-2.3.3/lib/net/http/persistent/ssl_reuse.rb:29:in `open'
from /home/c/.rvm/gems/ruby-1.9.2-p290@vcr_test/gems/net-http-persistent-2.3.3/lib/net/http/persistent/ssl_reuse.rb:29:in `block in connect'
from /home/c/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/timeout.rb:44:in `timeout'
from /home/c/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/timeout.rb:89:in `timeout'
from /home/c/.rvm/gems/ruby-1.9.2-p290@vcr_test/gems/net-http-persistent-2.3.3/lib/net/http/persistent/ssl_reuse.rb:29:in `connect'
from /home/c/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:637:in `do_start'
from /home/c/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/net/http.rb:632:in `start'
from /home/c/.rvm/gems/ruby-1.9.2-p290@vcr_test/gems/net-http-persistent-2.3.3/lib/net/http/persistent.rb:405:in `connection_for'
from /home/c/.rvm/gems/ruby-1.9.2-p290@vcr_test/gems/net-http-persistent-2.3.3/lib/net/http/persistent.rb:616:in `request'
from /home/c/.rvm/gems/ruby-1.9.2-p290@vcr_test/gems/mechanize-2.1/lib/mechanize/http/agent.rb:264:in `fetch'
from /home/c/.rvm/gems/ruby-1.9.2-p290@vcr_test/gems/mechanize-2.1/lib/mechanize.rb:319:in `get'
from vcr_playback_issue.rb:32:in `block in vcr_playback_issue'
from /home/c/.rvm/gems/ruby-1.9.2-p290@vcr_test/gems/vcr-1.11.3/lib/vcr/util/variable_args_block_caller.rb:8:in `call'
from /home/c/.rvm/gems/ruby-1.9.2-p290@vcr_test/gems/vcr-1.11.3/lib/vcr/util/variable_args_block_caller.rb:8:in `call_block'
from /home/c/.rvm/gems/ruby-1.9.2-p290@vcr_test/gems/vcr-1.11.3/lib/vcr.rb:76:in `use_cassette'
from vcr_playback_issue.rb:29:in `vcr_playback_issue'
from vcr_playback_issue.rb:37:in `<main>'

This is because newer versions of net-http-persistent (which mechanize depends on), net-http-persistent-2.3.3/lib/net/http/persistent/ssl_reuse.rb:29 issues a TCPSocket.open(conn_address(), conn_port()) that VCR doesn't catch. net-http-persistent adds SSL session reuse.

You can narrow the issue down to this which will try to connect to the internet even in 2.0.0.rc1

VCR.use_cassette("vcr_playback_issue") do
  TCPSocket.open("google.com", "443")
end

Even if VCR isn't supposed to handle socket requests, this prevents tests from completing in an offline environment so an alternative solution or at least a better error message would be helpful.

[myronmarston]

@christophermanning -- thanks, that's super helpful! I haven't used mechanize 2.1 or net-http-persistent so I had no idea about this issue.

Ultimately, I would consider this a bug in FakeWeb and WebMock; VCR, to stay abstract and high-level, delegates all Net::HTTP stubbing/connection-disabling to those libraries. VCR will never support code like:

VCR.use_cassette("vcr_playback_issue") do
  TCPSocket.open("google.com", "443")
end

...because that is interacting directly at the TCP socket layer, and VCR only deals with HTTP.

That said, I want to ensure this is fixed for VCR users so I will take a look at fixing it in WebMock (which I have commit access to) and reporting a bug to FakeWeb.

@pboling -- are you using net-http-persistent as well? That would explain why you have gotten this error but I haven't.

[pboling]

It doesn't appear that we are, though I would guess it is a similar issue with a different call. We have a heavily modified (patched) stack. I haven't been able to put any more time into it yet, but still hope to.

[dbalatero]

+1. It looks like I'm using net-http-persistent with the mechanize gem, and I'm not getting disabled HTTP errors as expected. Also, new cassettes are not being recorded.

# in spec/support/vcr.rb:
require 'mechanize'
require 'webmock/rspec'
WebMock.disable_net_connect!

require 'vcr'

VCR.configure do |c|
  c.cassette_library_dir = File.expand_path(File.dirname(__FILE__) + '/../fixtures/vcr')
  c.hook_into :webmock
  c.default_cassette_options = { :serialize_with => :json }
end

RSpec.configure do |c|
  c.extend VCR::RSpec::Macros
end

# in my spec:
use_vcr_cassette('mycassette', :match_requests_on => [:method, :host, :path])

[myronmarston]

@pboling -- I just added a feature to VCR that will give you some debug output to help troubleshoot what VCR's doing. It may help us identify your issue.

https://github.com/myronmarston/vcr/blob/master/features/configuration/debug_logging.feature

Can you give this a try and paste the output here?

[shayfrendt]

@myronmarston I ran into something strange today that feels related to this:

When using VCR, Typhoeus, and FakeWeb together with {:record => :new_episodes}, cassettes were not being written to a file.

When using VCR, Typhoeus, and WebMock together with {:record => :new_episodes}, cassettes were being written out to a file.

Thanks to your ever-helpful debug_logging feature, I was able to track it down....

# Gemfile

source :rubygems

gem 'vcr', :git => "https://github.com/myronmarston/vcr.git", :ref => "7824d07d0cbc52292742bd97801959e8330ca856"
gem 'fakeweb', '1.3.0'
gem 'typhoeus', '0.3.3'

I the ran the following test file with ruby vcr_test.rb:

# vcr_test.rb

require 'bundler/setup'
require 'vcr'
require 'fakeweb'
require 'typhoeus'

VCR.configure do |config|
  config.hook_into :fakeweb
  config.cassette_library_dir = "cassettes"
  config.default_cassette_options = {:record => :new_episodes, :match_requests_on => [:uri, :body, :headers]}
  config.debug_logger = File.open("playback.log", 'w')
end

VCR.use_cassette('example') do
  Typhoeus::Request.get("http://news.ycombinator.com")
end

This is the output of cat playback.log:

[Cassette: 'example'] Initialized with options: {:record=>:new_episodes, :match_requests_on=>[:uri, :body, :headers], :serialize_with=>:yaml}

There were no files under the "cassettes" directory.

I then ran ruby vcr_test.rb again, but first swapped out using Typhoeus for Net::HTTP:

VCR.use_cassette('example') do
  Net::HTTP.get_response(URI("http://news.ycombinator.com"))
end

Then the output of cat playback.log looked right:

[Cassette: 'example'] Initialized with options: {:record=>:new_episodes, :match_requests_on=>[:uri, :body, :headers], :serialize_with=>:yaml}
[fakeweb] Handling request: [get http://news.ycombinator.com/ "" {"accept"=>["*/*"], "user-agent"=>["Ruby"]}](disabled: false)
  [Cassette: 'example'] Initialized HTTPInteractionList with request matchers [:uri, :body, :headers] and 0 interaction(s): {  }
[fakeweb] Identified request type (recordable) for [get http://news.ycombinator.com/ "" {"accept"=>["*/*"], "user-agent"=>["Ruby"]}]
[Cassette: 'example'] Recorded HTTP interaction [get http://news.ycombinator.com/ "" {"accept"=>["*/*"], "user-agent"=>["Ruby"]}] => [200 "<html><head><link rel=\"stylesheet\" type=\"text/css\" href=\"http://ycombinator.com/"]

And the new cassettes were written out to cassettes/example.yml.

(I also swapped out FakeWeb for WebMock and was able to use WebMock with Typhoeus and Net::HTTP successfully.)

It seems like there's either a bug somewhere in FakeWeb, or that FakeWeb just doesn't play nice with Typhoeus. Ever seen something like this?

[myronmarston]

@shayfrendt -- thanks for the detailed bug report.

FakeWeb only supports stubbing Net::HTTP; thus, if you configure hook_into :fakeweb, VCR's not going to be able to support stubbing other HTTP client libraries such as Typhoeus. VCR has direct support for typhoeus; configure hook_into :fakeweb, :typhoeus and VCR will handle requests made using Net::HTTP.

WebMock supports a lot of different HTTP libraries (I think 6 or 7 at this point), including both Net::HTTP and Typhoeus, thus, VCR is able to handle requests made using Net::HTTP and Typhoeus when you only configure hook_into :webmock.

Does that make sense? Is there a way the documentation can be improved to make this more clear?

[shayfrendt]

@myronmarston The more you know. That definitely explains what I was seeing.

I think if I had read the hook_into documentation a little more carefully, I would have quickly seen this cause it's right there front and center.

Thanks for the lightning-quick reply!

[myronmarston]

For those of you having problems with net-http-persistent and/or mechanize, it looks like @drbrain was kind enough to fix it there:

https://github.com/drbrain/net-http-persistent/commit/614c0230b68649a18662f1a8746c69982ed0c6a2 https://github.com/drbrain/net-http-persistent/issues/13

I'm going to close this as the main problem people have been having has been solved and there hasn't been anything new posted to help me troubleshoot @pboling original issue. @pboling -- feel free to reopen this issue with more info if you figure out anything else or come up with an reproducible example I can play with.

[vrish88]

@myronmarston we are seeing the same issue with VCR 2.4.0, webmock 1.9.0, soap4r 1.5.8, and httpclient 2.3.2 where live requests are going through. We've tried reordering the various require statements and nothing appears to work.

Here's our vcr debugging output:

[Cassette: 'AdNetworks::ApprovalStatusJob'] Initialized with options: {:serialize_with=>:yaml, :match_requests_on=>[:method, :host], :persist_with=>:file_system, :record=>:new_episodes, :allow_unused_http_interactions=>true}

Any thoughts on why these interactions wouldn't be recorded?

[myronmarston]

Without an executable example to play with, it's impossible for me to say what the problem is. That said, if that's all the debugging output you're getting, it suggests that WebMock isn't hooking into the HTTP client requests at all. I believe that can happen when the HTTPClient instance is created before webmock is loaded. You should make sure that WebMock is loaded before the HTTPClient instance if you want WebMock to hook into it.

[vrish88]

There it is! Requiring WebMock before we loaded our app was the trick. Thanks a bunch and now it is time to enter a whole new beautiful world of testing :)