WindowSwitchingTest testCanCallGetWindowHandlesAfterClosingAWindow is flaky on Android

GoogleCodeExporter commented 9 years ago

Over a large number of runs, this test fails about 5% of the time. It is able 
to open a new tab, scroll to and click the "close" link, but chromedriver does 
not register the tab closing:

https://github.com/SeleniumHQ/selenium/blob/72ef60dd4da2b76f2aff5450fa88b34b10fb
d6b2/java/client/test/org/openqa/selenium/WindowSwitchingTest.java#L262

Original issue reported on code.google.com by samu...@chromium.org on 2 Mar 2015 at 6:08

GoogleCodeExporter commented 9 years ago

Some testing on Friday suggests that this might be a flakiness issue in the 
browser. Attaching a test case that attempts to reproduce this without 
ChromeDriver.

Original comment by samu...@chromium.org on 2 Mar 2015 at 6:11

Attachments:

GoogleCodeExporter commented 9 years ago

Testing with better page1.html and page2.html did NOT prove that this can be 
reproduced without using chromedriver's click(). In other words, it is possible 
(although not certain) that the way chromedriver does click or the way 
chromedriver waits for the page to be ready to be interacted with is 
problematic.

I ran this test some 20000 times using the Android tablet emulator (API level 
19) and it never failed to me:

*************** TEST (Java code) ****************************

  @Test
  public void test() throws Exception {
    driver = new ChromeDriver();
    WebDriverWait wait = new WebDriverWait(driver, 5);
    driver.get(server.locationOf("/page1.html"));

    // We have to click the link so that popup doesn't get blocked, see comment in page1.html.
    wait.withMessage("link didn't show up").until(presenceOfElementLocated(By.id("link"))).click();

    // Next line is commented outout because we have minimal delay (1ms) on page2.html before
    // we execute the script that closes the tab. Most of the time webdriver doesn't get to
    // catch the second page before it gets closed.
    // wait.withMessage("second page didn't show up").until(windowHandleCountToBe(2));

    wait.withMessage("second page didn't disappear").until(windowHandleCountToBe(1));
  }

  private static ExpectedCondition<Set<String>> windowHandleCountToBe(final int count) {
    return new ExpectedCondition<Set<String>>() {
      public Set<String> apply(WebDriver driver) {
        Set<String> handles = driver.getWindowHandles();

        if (handles.size() == count) {
          return handles;
        }
        return null;
      }
    };
  }

*************** page1.html ****************************
<!DOCTYPE html>
<html>
  <body>
    <a href="page2.html" id="link" target="result">the link</a>
    <script>
      // chrome-android driven by chromedriver always has popup blocker always ON due to
      // https://code.google.com/p/chromedriver/issues/detail?id=1021
      // So popups opened by a script on the page get blocked and we have use webdriver.click().
      // document.getElementById('link').click();
    </script>
  </body>
</html>

*************** page2.html ****************************
<!DOCTYPE html>
<html>
  <head>
    <script>
      function f() {
        var padding = document.getElementById('padding');
        padding.style.height = window.innerHeight * 1.5 + 'px';

        var link = document.getElementById('link');
        var box = link.getBoundingClientRect();
        var middle = box.top + box.height / 2;

        setTimeout(function() {
          window.scrollTo(0, middle);
          link.click();
        }, 1);
      }
    </script>
  </head>
  <body onload="f()">
    <div id="padding">pad ding</div>
    <a href="page2.html" id="link" onclick="window.close()">the close link</a>
  </body>
</html>

Original comment by vlotoshn...@gmail.com on 18 Mar 2015 at 2:45

ericmckean / chromedriver

WindowSwitchingTest testCanCallGetWindowHandlesAfterClosingAWindow is flaky on Android #1044