Thread hanging when used with Capybara.

theotherdon commented 9 years ago

When using pry-byebug to try to debug some feature tests, I've run into what I believe is a deadlock. Whenever I try to use Capybara's finder API my thread hangs indefinitely. I saw this deivid-rodriguez/pry-byebug#69 and tried removing pry-byebug to no avail. I did some digging and it looks to be with byebug itself. As per your request from the referenced thread, I'm creating this new issue in the byebug repo.

System Details

ruby 2.2.0p0 (2014-12-25 revision 49005) [x86_64-darwin14] rbenv 0.4.0 OSX Yosemite 10.10.5

Tested Versions

This issue sprung up somewhat recently (within the last month or so), but I've just worked around it. In light of this, I tried running byebug versions 7.0.0 up through the latest on master but the issue remained the same.

Reproduction

I've created a sample application with a minimal reproduction: https://github.com/donald-s/byebug_bug.

Clone the repo
Install Qt 5 (if you don't already have it) for capybara-webkit
Run bundle install
Run rspec spec/features/home_spec.rb and watch it pass (ensuring no issues with Capybara)
Finally, uncomment byebug , run the test again with the above command, and run page.find('h1') when the debugger opens.
Expected Behavior

Capybara returns an object or raises an error.

Observed Behavior

Thread running byebug hangs indefinitely.

I removed byebug and pry-byebug from the repo I originally discovered this in and things work as expected.

deivid-rodriguez commented 9 years ago

Hi @donald-s ! Thank's for the great report. I'll have a look when I get time. It will most likely not be very soon, but I'll get to it.

theotherdon commented 8 years ago

Hey @deivid-rodriguez!

You're welcome. Thanks for all of the work you've put into this project! Byebug has been an invaluable resource to me as a Ruby developer.

No problem. Totally understand. As a matter of fact, I'd love to have a crack at fixing this. If you don't mind telling me, what is your process for debugging byebug? (I can't help but make a reference to inception here. ;))

os97673 commented 8 years ago

I'm not sure about @deivid-rodriguez but I usually use debug output :) As for the problem I suspect the call tries to perform some operation on some dedicated thread which is stopped by byebug.

deivid-rodriguez commented 8 years ago

@donald-s That's great to hear!

I often use debug output, just like @os97673 or gdb when dealing with the c-extension part. Also very useful for developing is setting Byebug.verbose = true that makes the TracePoint API event handlers dump information about every event that is processed.

theotherdon commented 8 years ago

@os97673 Thanks! By "debug output" do you mean just printing to the terminal using puts or pp?

I belive you're correct. I did some digging in Capybara and found that the code hangs on the synchronize call in Capybara's find method here.

theotherdon commented 8 years ago

@deivid-rodriguez Perfect. I was wondering if there was a Byebug verbose option or something like that. Thanks!

os97673 commented 8 years ago

Thanks! By "debug output" do you mean just printing to the terminal using puts or pp?

yep. Verbose mode is useful but may be too noisy.

theotherdon commented 8 years ago

Got it. I'm going to do some digging as soon as I get a chance. Thanks for you help!

mikz commented 8 years ago

Hey, posted my findings in the other issue https://github.com/deivid-rodriguez/pry-byebug/issues/69#issuecomment-173876843. Imho it is surely bug in byebug (or an expected behaviour). Capybara webkit tries to connect to a socket in a thread to do #inspect on a object. Looks the same as https://github.com/deivid-rodriguez/byebug/issues/115

deivid-rodriguez commented 8 years ago

Yes, bug is acknowledged! :)

theotherdon commented 8 years ago

Hoping to provide a fix shortly.

deivid-rodriguez commented 8 years ago

That's sooo cool!! Thanks!

deivid-rodriguez commented 8 years ago

@donald-s Should I assume you gave up?

theotherdon commented 8 years ago

Hi Deivid, sorry for the delay in getting to this. I've been swamped. It may be a couple of more weeks before I'm able to get to it. I'd still love to take a crack at it if that works.

deivid-rodriguez commented 8 years ago

@donald-s Any news?

theotherdon commented 8 years ago

@deivid-rodriguez Yes. I finally got a chance to look into this. I've narrowed down the hang to an IO#select call that reads from a socket in capybara-webkit here. I'm trying to get a chance to dig deeper on capybara-webkit's side. Does this spark any ideas off the top of your head on the byebug side?

deivid-rodriguez commented 8 years ago

@donald-s I guess the thread that is supposed to write to that socket is blocked by the debugger... so the IO.select hangs there.

theotherdon commented 8 years ago

Yes. I'm trying to figure out where a good spot to start looking in byebug would be. Any ideas?

dkniffin commented 8 years ago

Has there been any more progress on this? I've been seeing this issue for a while now, and would love to have it fixed.

deivid-rodriguez commented 8 years ago

Not that I know of.

theotherdon commented 8 years ago

@dkniffin The farthest I've been able to get so far is figuring this out is that the hang it happening in capybara-webkit on an IO select call. I'm not sure if this is an issue on capybara-webkit's side or byebug's (or both). I'm at a bit of a loss as to where to dig in on capybara-webkit's side. Any ideas?

dkniffin commented 8 years ago

@donald-s Hmm. I don't. Could you give some more details about how you got that far? I'm not too familiar with the internals of byebug or capybara-webkit.

theotherdon commented 8 years ago

@dkniffin Sure. capybara-webkit communicates with the capybara server via TCPSockets. After a lot of putss, I traced this thread hanging issue down to an IO#select call in capybara-webkit's Connection class here. IO select receives an array of IO objects and waits until one or more of them is ready for reading, writing, or raises an exception. In capybara-webkit's Connection class, there's a Thread.join waiting for IO.select to complete. For some reason, the socket is not returning anything, so IO.select waits indefinitely (because no timeout is supplied). I'm trying to figure out why the socket isn't returning anything.

goooooouwa commented 8 years ago

@donald-s any clue yet?

theotherdon commented 8 years ago

@goooooouwa Still working on it. I'm digging through capybara-webkit's code to get a better understanding of how their Connection class is working.

deivid-rodriguez commented 8 years ago

Just wanted to thank @donald-s for all the research and effort put in here. :heart: :yellow_heart: :purple_heart:

theotherdon commented 8 years ago

Thanks @deivid-rodriguez! Hoping I'll be able to come up with something shortly here. I'm doing more digging on this today.

theotherdon commented 8 years ago

@deivid-rodriguez After more digging, I've determined that execution control is not being passed to capybara-webkit at all. What parts of byebug handle IO? I'm almost positive the issue is that byebug is somehow intercepting the IO before it hits the capybara-webkit driver.

deivid-rodriguez commented 8 years ago

@donald-s Not sure it will help, but you can have a look at lib/byebug/interface*.

theotherdon commented 8 years ago

@deivid-rodriguez That was helpful, thank you. I'm trying to find somewhere that I can throw puts in byebug to see why it's not passing execution to capybara-webkit. After more digging, I ended up at CommandProcessor. I see that the #repl command continues until @proceed is true. Can you tell me a little bit more about what's going on here? How does byebug actually pause execution? Is this in the C extension?

deivid-rodriguez commented 8 years ago

That @proceed variable is flagged whenever normal execution should be resumed. For example, from next, continue or step commands (as opposed to break or display commands which leave control straight back to the user).

Execution is paused, indeed, from the C-extension. When certain conditions are fulfilled (for example, when the current line has a breakpoint enabled), the C-extension runs a callback method that takes us back to ruby land and gives the user back a prompt (those are the at_* methods in lib/byebug/context.rb).

Hope it helps.

surzycki commented 8 years ago

It appears to work with remote debugging...

theotherdon commented 8 years ago

@deivid-rodriguez Thanks. I'm going to throw some puts as soon as I have a chance to work on this more.

deivid-rodriguez commented 8 years ago

@donald-s Thanks to you, I wish I had more time to properly investigate this... :disappointed:

taylorzr commented 7 years ago

Happy new year! Any updates here?

taylorzr commented 7 years ago

Here's my two cents: We were using pry-byebug and capybara at some point in the past and it was not hanging. And then one day, we noticed it was hanging. If this is a byebug bug, I would assume some change in byebug caused this problem. So I did some history bisecting.

I cloned @donald-s repo, and updated the gemfile to use older versions of byebug. However, I noticed this hanging still occurs in older versions of byebug, even all the way back to byebug 1.0.1. I forked @donald-s repo, and created an example branch.

So I figured maybe it's not byebug, maybe capybara or capybara-webkit caused this problem. However, this hanging still occurs with a fairly old version of capybara & capybara-webkit. I also created an example branch for this.

So... :man_shrugging:. Maybe it's a different gem? Maybe it's a binary that one of these gem's uses?

System Details

OSX Yosemite 10.10.5 rvm 1.28.0 (latest) ruby 2.2.0p0 (2014-12-25 revision 49005) [x86_64-darwin14] also tried ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-darwin14]

deivid-rodriguez commented 7 years ago

Sorry, forgot to answer to this: no news.

You have to also play with ruby versions in your investigations. Byebug uses a relatively new API for debugging that was first introduced in 2.0.0 and came with some bugs. In the README you can find the ruby versions recommended for usage with byebug. For example, 2.2.0 is not recommended because it has bugs fixed in later patch level releases.

theotherdon commented 7 years ago

@taylorzr Sorry for the delay in getting back to you. I've unfortunately still not had any time to do more digging on this. It sounds like you're onto something with this possibly not being in byebug. The best conclusion that I've been able to come to up to this point is that something is causing some TCP sockets in capybara-webkit not to receive the data that they need which is resulting in the hang.

I also haven't tried other Ruby versions like @deivid-rodriguez mentioned. I must have missed the part where it said not to use Ruby 2.2.0. 😬

phoet commented 7 years ago

i just ran into this issue, it drove me nuts for half a day 😭

my observations are:

ruby versions do not matter
rails versions do not matter
pry-rails works as expected
thread unblocks when calling visit ...

the last thing is quite funny. once you call visit in the byebug repl, the request goes through! this is reproducible and i used it as a crutch to debug my capybara/poltergeist test.

since i really need this debugging feature i will have to drop byebug until this is fixed 😢

deivid-rodriguez / byebug