Wrong order of executed statements

[philly-mac]

Hi,

First off I just want to say that I am not sure whether to post this issue here, or on the sequel page, as I am not sure wherein lies the problem.

So here is what I am doing

I am using padrino + bacon + sequel + capybara + selenium | capybara-webkit to run tests.

When doing this with selenium, I get the output and result that I expect. See the following gist.

This snippet of code

https://gist.github.com/1504690

produces this out put

https://gist.github.com/f9a9a5bc122ff01d10f2

but with capybara-webkit I get this output

https://gist.github.com/42b9bee8347a7cbf5030

As you can see, the webkit output shows that when the button is clicked, the actions that happen within the controller are executed after all the actions in the test itself.

This seems very strange to me, as

1. The same setup works with selenium and browsing the code I haven't seen anything specific to transactional fixtures.
2. I am not using transactions for these tests at all

Now, as I said before I am not sure if this is a sequel issue or a capybara-webkit one. But seeing as I would prefer to use webkit over selenium, I thought I would try here first. I'll bug the sequel guy/s later if I get no luck here :)

Any ideas?

[jeremyevans]

Sequel maintainer here.

You mentioned earlier to me that capybara-webkit was using a different thread (http://groups.google.com/group/sequel-talk/browse_thread/thread/dcfab7dcdbee9780), maybe if you join that thread after the click_button, things may work? If I had to guess, this is a race condition.

I use Sequel with capybara with no issues, but not capybara-webkit, so this is outside of my area of expertise. This does not appear to be a Sequel issue, though.

[philly-mac]

@jeremyevans Well, I was just going off of what is said in the capybara documentation here

https://groups.google.com/forum/#!msg/ruby-capybara/JI6JrirL9gM/R6YiXj4gi_UJ

The most important part is

What happens under Selenium is this: Capybara starts up your Rails app
using a webserver (usually webrick or thin) in a *separate thread* of
the *same process*. 

I will see if there is a simple way for me to join thread after the click. If that doesn't work, I will defer to the knowledge of the capybara-webkit developers.

[philly-mac]

So joining the threads just hangs the process. So I am guess that is no real solution.

Any other ideas guys?

[leehambley]

/cc (sounds like a good bug!)

[jeremyevans]

Well, there's always sleep 1 after the click_button. :)

[philly-mac]

Tried that (even with 5 seconds) still hung.

I wish there were an easy way to see how this is being executed compared to selenium.

[jferris]

I don't think this has anything to do with sequel. Most of capybara's Javascript drivers execute asynchronously. However, various drivers have found ways to automatically wait until some operations are complete so that some actions seem to execute synchronously. It seems like click_button in the Selenium driver's implementation executes synchronously.

Joining the threads doesn't make sense, because the background thread keeps running. It's an application server (generally WEBrick or Thin) running in a loop, so joining the thread will wait forever.

Capybara comes with a helper called wait_for that you can use to sync up the two threads. It takes a block that should return true once you know the action you're waiting for is complete. As an example, if you're expecting an action to create a new user, you could do something like this:

original_count = User.count
click_button "Signup"
wait_for { User.count == original_count + 1 }

That would get the original count, start processing the signup action, and repeatedly check the user count until it lined up. It will time out and raise an exception after two seconds by default if the condition is never met.

In general, I'd recommend testing things through the UI if you can anyway, so rather than checking that a user was created, I'd check that the page contains something that indicates the user was created. Helpers like have_css automatically use wait_for under the hood, so you don't have to worry about async issues if you use those.

[leehambley]

@philly-mac for what it's worth @jferris' advice is golden here, wait_for{} is evil, but have_css (waits for the CSS to arrive) and have_no_css (wait for it to disappear) are exceptionally useful. I'm using them a lot in my pet project.

[philly-mac]

Cheers guys.

Those tips seemed to work with getting things in the correct order, but now when I run my tests I get this error

Capybara::Driver::Webkit::WebkitInvalidResponseError: Unable to load URL: http://127.0.0.1:49089/bookings/427

Not sure what this means. Any ideas on this?

[jferris]

That means that either your application responded with an error or you found a bug in capybara-webkit. I'd look through your test.log to see if there's anything interesting about that request.

[philly-mac]

Test log shows absolutely nothing out of the ordinary. In fact I was getting those errors before, on every test run, I just thought it was the order of execution that was causing it to do that. But now it is still doing it which is why I am not sure what is causing it.

[philly-mac]

So upon further investigation it seems as if the problem may lie with padrino/sinatra. The tests that I had before just tested to see if the url I was at matched a certain regex, and it did. But even when run using selenium nothing is loaded into the page, and I guess this is where the webkit driver is also reporting an error saying that it cannot load the page.

So I am guessing that if I can find out what is stopping the page from loading even in selenium, the webkit driver will work as expected.

Thanks for all the help guys, I will do some more investigating or bug the padrino guys for more help on this one.

[philly-mac]

Okay final word on this, and my sanity saved just in time for Christmas. Turns out that the server which was being spun up was silently failing. And there was some error in the code, but I was just never seeing it.

After looking at this thread

https://github.com/jnicklas/capybara/issues/329

I added this code

Capybara.server do |app, port|
    require 'rack/handler/webrick'
    Rack::Handler::WEBrick.run(app, :Port => port, :AccessLog => [], :Logger => WEBrick::Log::new(Rails.root.join("log/capybara_test.log").to_s))
  end

I was actually able to see what was going on and where, and finally able to get things working again.

Silently swallowing up errors in testing is not a good thing at all. But glad I got it sorted, and maybe someone else that reads this may not have to go through this also.

[dantswain]

@philly-mac What was the WEBrick error you were seeing and how did you resolve it? I think I'm experiencing a similar issue.

[philly-mac]

@dantswain The error was not with webrick itself. It was with the code. But previously I was using thin to run locally, and it was swallowing up the errors and not reporting anything. So all I would know is that something wasn't working, but I had no idea why, or what. when I started using webrick it then started showing me the errors that thin was masking, allowing me to get everything working as it should.

[TALlama]

Using @jferris' suggestion works great, with the exception that if your controller throws any errors you get no stack trace but only an inscrutable:

Unable to load URL: http://127.0.0.1:54473/model/1 (Capybara::Driver::Webkit::WebkitInvalidResponseError)

You can jump into the log to see the issue, but I was wondering if there is there a good way around that, like a convenient:

page.should_not be_an_invalid_response_so_show_me_a_stacktrace_or_the_page_source

[jferris]

@TALlama this is a general issue with asynchronous capybara drivers. Depending on your Rails settings, either a development 500 page is displayed to the headless browser that you don't see, or it bubbles up to thin/webrick and just goes in the log. Either way, the feedback isn't great.

I started work on a branch to fix this here: https://github.com/thoughtbot/capybara-webkit/tree/reraise_exceptions

There are still some issues, but you can try that out or contribute if you'd like. Right now the tests in that branch are hanging and I haven't figured out why.