Selenium Java API does not conceal Action protocol differences with moveToElement(element,xOffset,yOffset)

[pbi-qfs]

Meta -

OS: Any Selenium Version: 3.6.0 Browser: Chrome, Firefox

Expected Behavior -

Selenium should do what its Javadoc says, and it should be consistent independently from the underlying wire protocol: moveToElement(element,xOffset,yOffset) moves the mouse to an offset from the top-left corner of the element.

Actual Behavior -

It prints out "INFORMATION: When using the W3C Action commands, offsets are from the center of element" and moves the cursor differently than stated in the method doc

Steps to reproduce -

final boolean useW3C = true;
final WebDriver driver = useW3C ? new FirefoxDriver() : new ChromeDriver();

driver.get("https://www.w3schools.com/jsref/tryit.asp?filename=tryjsref_event_mouse_clientxy");

driver.switchTo().frame("iframeResult");

final WebElement header = driver.findElements(By.tagName("h2")).get(0);

final int xOffset = 10;
final int yOffset = 5;

final Actions builder = new Actions(driver);
builder.moveToElement(header, xOffset, yOffset); // Javadoc: Moves the mouse to an offset from the top-left corner of the element.
builder.click();
builder.perform();

final Point location = header.getLocation();
final String expectedResult = String.format("X coords: %d, Y coords: %d",
        location.x + xOffset,
        location.y + yOffset);

final String result = driver.findElement(By.id("demo")).getText();

if (expectedResult.equals(result)) {
    System.out.println("fine.");
} else {
    System.out.format("not ok: '%s' instead of '%s'\n", result, expectedResult);
};

[JamesZoft]

In the w3c spec (https://w3c.github.io/webdriver/webdriver-spec.html#pointer-actions) it says:

Let x element and y element be the result of calculating the in-view center point of element.

In subsection "When required to dispatch a pointerMove action with arguments source id, action object, input state and tick duration a remote end must run the following steps"

So the docs are wrong.

[pbi-qfs]

The problem with moveToElement+offset is the spec difference between https://github.com/seleniumhq/selenium/wiki/jsonwireprotocol#sessionsessionidmoveto and https://w3c.github.io/webdriver/webdriver-spec.html#pointer-actions:

"…offset to move to, relative to the top-left corner…" vs "Let x element and y element be the result of calculating the in-view center point of element. Let x equal x element + x offset, and y equal y element + y offset. "

That should be transparent to the users of the Selenium API

[pbi-qfs]

The ruby bindings currently make up for the difference by shifting the position by the current element size (pointer_actions.rb#90), but as stated in Actions.java#409, we can't be sure that the element still has the same dimension at execution time as during action construction.

So, maybe instead of fixing it on the client side, we should align the W3C spec to the OSS protocol and shift the position in geckodriver (https://github.com/mozilla/geckodriver/issues/789) when matching the W3C action to the corresponding marionette command...

[JoolsCaesar]

This could really do with fixing (either in selenium or the docs). The majority of drivers are now W3C compliant. Chrome made the switched just over a month ago. The new Edge (Chromium) driver follows the W3C spec too.

[Rinzwind]

This could really do with fixing (either in selenium or the docs).

It seems the documentation has been changed: https://github.com/SeleniumHQ/selenium/commit/43963b8df7a04ee75c40425596c93ed63c6042c6

Note though that the documentation was changed to say “offset from the center of the element”, while as quoted by JamesZoft above, the W3C spec actually says it’s an offset from the “in-view center point of [the] element”:

In the w3c spec (https://w3c.github.io/webdriver/webdriver-spec.html#pointer-actions) it says:

Let x element and y element be the result of calculating the in-view center point of element.

A definition of an “element’s in-view center point” is given in section 12.1 “Interactability” in the spec.

As for the workaround that is applied in Ruby: as far as I understand, it translates a top-left corner offset to an offset from the element’s center rather than the element’s in-view center, so it requires that the element is completely in view.

[Rinzwind]

As for the workaround that is applied in Ruby: as far as I understand, it translates a top-left corner offset to an offset from the element’s center rather than the element’s in-view center, so it requires that the element is completely in view.

For those interested, in Parasol (WebDriver package for Smalltalk), we applied a workaround that better takes into account elements that are not completely in view: see the Javascript function that is used in the method BPActions>>#moveToElement:offset:.

[diemol]

After following the conversation, it seems the ambiguity went away. However, if this is still an issue with the docs, please raise a PR to update them and we will be happy to get it merged.

For now, since we are doing a issue cleanup, I will close this.

[Rinzwind]

@diemol: just a quick response, I see that the documentation still says “offset from the center”:

https://github.com/SeleniumHQ/selenium/blob/c7dd3d3570b3f117225acd7353cbf7cf95a2686f/java/client/src/org/openqa/selenium/interactions/Actions.java#L372-L375

Shouldn't that be changed to “offset from the element’s in-view center point”? See my earlier comment (Sept. 11th, 2019).

[diemol]

Perhaps I misread it, would you like to send us a PR, @Rinzwind?