search

rubycdp / cuprite

Headless Chrome/Chromium driver for Capybara
https://cuprite.rubycdp.com
MIT License
1.25k stars 92 forks source link

Flapping specs caused by timing issues with AJAX #139

Open matpowel opened 4 years ago

matpowel commented 4 years ago

So in general we've been really enjoying using this project. The biggest outstanding items are lack of drag/drop support for system tests around drag/drop in our app and then flapping specs. For the flappers, they seem to be caused primarily by timing issues which we never get in old Selenium branches.

We use the table widget from bootstrap-table.com fairly heavily in the app and load all the data using AJAX. Most of our flappers seem to center around Cuprite not correctly waiting for the Ajax requests that Bootstrap Table fires off to complete and the table to populate, so we get asserts failing intermittently. The only solution I've found so far is to add a "sleep(0.5)" after the click that fires the table AJAX reload and before the assert, which is stinky. This is never needed in modern iterations of Selenium.

Note that the fails only seem to happen on Heroku CI.

We're on:

route commented 4 years ago

I'm happy to debug and fix them all if you show me your tests, debug logs and collaboration :) Though I understand it should be done privately so my email is in my profile!

caseyprovost commented 3 years ago

I am seeing this behavior as well. Do we have any fix available or methodology of writing ajax-based tests so that they work more reliably?

matpowel commented 3 years ago

@caseyprovost I never found any, but please share if you find good fixes. We had to switch back to Selenium for now but we really would like to ride as close to native driving of browser as possible.

@route I hope we'll have time in the future to transpose all the issues to publicly shareable tests, but for now it's really quite simple, that is Cuprite doesn't seem to really have any native ability to detect and wait for outstanding Ajax requests. Is it supposed to? ie. is that functionality implemented and just not working sometimes? So for example as mentioned in my description, if you use the bootstrap-table.com component and trigger table refresh programmatically and then try to match the DOM with Capybara it fails, we needed to sprinkle sleep(0.5) throughout the tests. Selenium doesn't require any sleeps.

I'm super thankful for all your hard work on this project and not critical at all, you seem to have done so much of it on your own and that's a huge job. But we're an early stage, bootstrapped startup and need to focus on feature code for now. We'll have time to contribute to various projects more later in the year I think.

route commented 3 years ago

@caseyprovost nothing fancy just properly write capybara tests and you should be good.

https://thoughtbot.com/blog/write-reliable-asynchronous-integration-tests-with-capybara https://evilmartians.com/chronicles/system-of-a-test-setting-up-end-to-end-rails-testing

route commented 3 years ago

@matpowel Like I said I'm happy to help and debug, so please if you come up with a simple test that reproduces the issue, I'll investigate, as for AJAX there are methods like https://github.com/rubycdp/ferrum#wait_for_idleoptions and so on to help people working with it