Support xpath support for cy.get() and .find()

[fringd]

Current behavior:

.get('//div') doesn't work .find('p/span') also doesn't work

Desired behavior:

It gets the element using document.evaluate just like get and find currently uses document.querySelector.

or to avoid collisions we add getXpath and findXpath or something.

Motivation

xpath is fairly standard in integration tests, and is much more powerful than css selectors are, and I'm working on a shared library of xpath selectors for use with selenium, puppeteer, and ideally, cypress.

I'm happy to help with writing this code if people agree it's a useful feature that would be likely to get merged.

[jennifer-shehane]

I don't know the full depth of the complexity of the implementation of this, but this is definitely possible.

• cy.get() code is custom to cypress: https://github.com/cypress-io/cypress/blob/issue-895/packages/driver/src/cy/commands/querying.coffee#L93
• .find() code just passes is along to jQuerys implementation basically: https://github.com/cypress-io/cypress/blob/issue-895/packages/driver/src/cy/commands/traversals.coffee#L5

[3d-mac]

+1 for XPath support

[mgalindo]

+1

[CliffDavis]

+1, this would make migrating from an existing selenium framework significantly easier.

[brian-mann]

These things already exist. You can use off the shelf tools to convert xpath to css selectors.

https://www.google.com/search?q=xpath+to+selector+converter&oq=xpath+to+selector+converter&aqs=chrome..69i57j0l4.6500j0j4&sourceid=chrome&ie=UTF-8

There are a few NPM modules you can just install and require in cypress.

[fringd]

I'm pretty sure xpath is a super-set of css, so any translator from xpath to css would be incomplete. Part of the reason I want xpath is that it's more powerful.

[fringd]

for example, in xpath you can select a node depending upon its children, or its text content.

[JashonBrown]

+1

[JMVL64]

+1

[dharshinid]

+1

[ravitheja04]

+1

[ailadson]

+1, although i got what I needed with cy.get('svg').find('id-of-svg-element')

[moxventura]

+1

[nusco]

+1. Even in cases where CSS is enough, having Xpath would make some migrations from other frameworks so much easier.

[szymach]

Yeah I've just ran into an issue, where a select() did not work due to a non-braking space in the inner HTML of the option. In XPath I could just normalize spaces and it would work just fine, so definitely a +1 from me.

[saintflow47]

+1. Adding a data-cy attribute is a nice way, but we should be allowed to use xpath as well if we don't want to add any code specifically for testing.

[bryantabaird]

Yes, searching by text content would be great with xpath, since that cannot be done with CSS selectors (it doesn't allow contains, see https://stackoverflow.com/questions/1520429/is-there-a-css-selector-for-elements-containing-certain-text?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa)

Edit: just found this: https://docs.cypress.io/api/commands/contains.html#Content

[eduhlana]

+1

[kamituel]

I'd love that. I use CSS selectors whenever possible, but sometimes XPath is necessary.

One example of a selector that is hugely useful in XPath, but not possible in CSS, is selecting a table cell based on the header.

For instance, to get a cell in row 3 in column with a label "Age", you can use:

//table
  /tbody
    /tr[position()=3]
      /td[count(//table/thead/tr/th[text()='Age']/preceding-sibling::th) + 1]

Not exactly a trivial selector, but I still prefer it over doing this programmatically.

Here's a corresponding table:

<table>
  <thead>
    <tr>
      <th>Username</th>
      <th>Email address</th>
      <th>Age</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Mary</td>
      <td>mary@doe.com</td>
      <td>62</td>
    </tr>
    <tr>
      <td>John</td>
      <td>john@doe.com</td>
      <td>60</td>
    </tr>
    <tr>
      <td>Kate</td>
      <td>kate@doe.com</td>
      <td>20</td>
    </tr>
  </tbody>
</table>

[v-mwalk]

Am averse to 'me toos' and +1's but xpath capability it a must. There are a few scenarios where css will not work; the primary one being pointed out by fringd.

Even if it is the XPath 1.0 engine, it would make Cypress fully usable for any webpage...

[brian-mann]

@kamituel in your example above why couldn't you just do this?

// find the cell containing the content
cy.contains('td', '20')

// or you could find a higher up element if you wanted to
cy.contains('tr', '20')

The vast majority of use cases I've seen are for XPath related to an element containing text and you can accomplish that with cy.contains().

[brian-mann]

I just looked it up and apparently there is already a document.evaluate which will take an xpath.

You should already be able to do this with Cypress with a custom command.

https://developer.mozilla.org/en-US/docs/Web/API/Document/evaluate

At the very least, adding this to the query methods of Cypress may not be that difficult afterall.

[brian-mann]

@bryantabaird went back through to review comments. Per my comment here: you can use cy.contains() rather than xpath if you want to select an element or a parent element by its text content.

We've found cy.contains() can achieve what you normally would do with a much more complex xpath selector.

https://github.com/cypress-io/cypress/issues/1274#issuecomment-403538946

[kamituel]

@brian-mann To follow a real example, I'm working on a web application that has several large tables (for reporting purposes), that can easily have 20-30 columns. In some test cases, I want to assert on contents of just a few out of those 20-30 columns.

Using: cy.contains('td', '20') would work, but:

• it doesn't specify in which column I expect to see the value "20"
• it leads to not very readable test code that will grow to be hard to maintain

Let's assume, still following my example, that I want to assert on contents of the first and third column, and I'm not interested in the contents of the 2nd:

cy.contains('td', '20')
cy.contains('td', 'Kate')

This code will match:

    <tr>
      <td>Kate</td>
      <td>kate@doe.com</td>
      <td>20</td>
    </tr>

but it'll also match:

    <tr>
      <td>20</td>
      <td>kate@doe.com</td>
      <td>Kate</td>
    </tr>

So I can start adding some more specific selectors, or use data-cy:

cy.contains('td:nth-of-type(3)', '20')
cy.contains('td:nth-of-type(1)', 'Kate')

But it gets tricky quickly - I'll have to update tests as the table grows, or when I remove a column, etc...

Compare it to (a hypothetical code):

cy.get(cell(3, 'Age')).contains('20');
cy.get(cell(3, 'Username')).contains('Kate');

// With a helper function:

function cell(rowNumber, columnHeader) {
  return `//table
            /tbody
              /tr[position()=${ rowNumber }]
                /td[count(//table/thead/tr/th[text()='${ columnHeader }']/preceding-sibling::th) + 1]`;
}

Sure, the helper function uses a fairly complex XPath selector, but it's not that bad once one gets used to XPath, and it's still simpler than a helper function/command would be without XPath. Plus, it leads to simple, readable and maintainable test code.

In general, I prefer CSS over XPath any day, as it leads to shorter selectors whose syntax everyone is familiar with. But when given a choice of a complex helper function that would need to iterate over a number of DOM nodes, versus a declarative XPath selector, I strongly prefer XPath.

Also, yes, there is document.evaluate I use often to test XPath selectors (when using them with Selenium). I even added a bookmarklet to Chrome which allows me to easily use it on every website:

javascript:(window.find_all = function (query) {     var nodes = [];     var iterator = document.evaluate(query, document);     try {         var node = iterator.iterateNext();         while (node) {             nodes.push(node);             node = iterator.iterateNext();         }     } catch (e) {         console.error("Document tree modified during iteration", e);         return null;     }          return nodes; })();

Usage:

find_all(xpath_selector);

I would imagine in Cypress, we could accept XPath selectors prefixed with some character. Like in cy.get(...), which accepts aliases with @ prefix. Or maybe we could just use /?

[brian-mann]

What I'm saying is that with Cypress you already have native DOM access to your window/document.

Just use the code you wrote just now and put it in a custom command so that it finds you the DOM element and then pass that off to cy.wrap() so you can then assume the element and then continue to chain off of it via Cypress.

[kamituel]

You mean write a custom command that'll accept an XPath selector and yield a DOM element? Sure, we can do that. And I'm pretty sure I eventually will, as we slowly (but surely) go about migrating our Selenium tests to Cypress ;)

Question is - should Cypress' built-in .get() support XPath? (or maybe have support for it in a separate command...)

I'd estimate that maybe 5% of selectors I would write in Selenium are XPath. Apparently, 20+ people who upvoted the OP have somewhat similar experiences. Is it enough to warrant having it built into Cypress itself? Not for me do decide :)

If you, the Cypress' core team, will decide it's not worth the effort, would you accept a PR that adds XPath support to .get(), or you rather think it belongs elsewhere (an external library)? Do you have plans to provide a "commons" library with stuff that is helpful to a large number of people, but doesn't necessarily belong to the set of core API's?

[jennifer-shehane]

@kamituel We would certainly accept a PR that adds support for XPath! We are not against supporting XPath considering the demand from users here.

Unfortunately, we are a small team, so it is not within our roadmap at this moment to work on this - so @brian-mann suggestions were to help you all with a workaround to use now.

We also feature custom commands in our plugins, so if that's an easier option than doing a PR - let us know and we can share it there.

[HGani1]

Another +1 for Xpath support in the Cypress .get() method

[gabriel-c-pereira]

+1

[eugene-ray]

any updates on this? -it's pretty useful in terms of work with the pretty complex DOM

[janusqa]

Hi @brian-mann

I am having trouble understanding what you are saying.

I have this...

1.

function getElementByXpath(path) {
  return document.evaluate(
    path,
    document,
    null,
    XPathResult.FIRST_ORDERED_NODE_TYPE,
    null
  ).singleNodeValue;
}

and can do this...

getElementByXpath('//input[@type="text" and @name="login"]');  //returns a node

What do I need to do in cypress to get the equivalent of cy.get(selector).type('blah') but using the above? Is it possible?

[kamituel]

@facebookdevbim The idiomatic approach would be to either add a new, custom command, such as cy.getXpath(...), or override built-in cy.get() to accept XPath expressions.

I opted for the latter approach myself:

Cypress.Commands.overwrite('get', (originalGet, selector, options) => {
  if (typeof selector === 'string' && selector.startsWith('#xpath#')) {
    var nodes = [];
    var iterator = document.evaluate(query, document);
    try {
       var node = iterator.iterateNext();
       while (node) {
         nodes.push(node);
         node = iterator.iterateNext();
       }
    } catch (e) {
      console.error("Document tree modified during iteration", e);
      return null;
    } 
    return nodes;
  } else {
    return originalGet(selector, options);
  }
})

I'm using it like this:

cy.get('#xpath#//div')

As you can see, I opted for having a prefix #xpath# before every XPath selector. You could do without it and for instance rely on the // too, probably.

Note: consider the code above pseudocode, as I didn't run it. I translated my code to JS by hand just to illustrate the approach.

[janusqa]

Hi again @kamituel

I modified your suggestion a little bit, but am unable to get it to work, so I need a little more assistance.

See below...

The return value always seems to be [] an empty array.

Cypress.Commands.overwrite("get", (originalFn, selector, options) => {
  if (typeof selector === "string" && selector.startsWith("XPATH")) {
    let nodes = [];
    let xPathExpression = selector.replace(/^(XPATH)/, "");
    let xPathDOM = document.evaluate(
      xPathExpression,
      document,
      null,
      XPathResult.ORDERED_NODE_SNAPSHOT_TYPE,
      null
    );
    for (let i = 0; i < xPathDOM.snapshotLength; i++) {
      nodes = nodes.push(xPathDOM.snapshotItem(i));
    }
    return nodes;
  } else {
    return originalFn(selector, options);
  }
});

Some questions...

1. the testable app is enclosed in an iframe, will that cause any issue when overwriting the built in "get"?

2. Isnt get returning some type of cypress chainable result? When we return nodes, are we returning a correct result that is comparable to what the original "get" was returning? From my understanding built into cy functions are all sorts of wizardy like waiting until a element appears ect. Are we bypassing all this by overwriting the built-in "get"?

[janusqa]

Managed to get it working by using cy.document in the custom command. Not sure why I needed it though.

"use strict";

// helper functions
let getElementByXpath = function(xPathExpression, container = document) {
  return document.evaluate(
    xPathExpression,
    container,
    null,
    XPathResult.FIRST_ORDERED_NODE_TYPE,
    null
  ).singleNodeValue;
};

let getElementsByXpath = function(
  xPathExpression,
  container = document
) {
  let xPathResult = null;
  let element = null;
  let xPathDOM = [];

  xPathResult = document.evaluate(
    xPathExpression,
    container,
    null,
    XPathResult.ORDERED_NODE_ITERATOR_TYPE,
    null
  );

  try {
    element = xPathResult.iterateNext();
    while (element) {
      xPathDOM.push(element);
      element = xPathResult.iterateNext();
    }
  } catch (e) {
    return null;
  }

  return xPathDOM;
};

let getElementsSnapshotByXpath = function(
  xPathExpression,
  container = document
) {
  let xPathResult = null;
  let xPathDOM = [];

  xPathResult = document.evaluate(
    xPathExpression,
    container,
    null,
    XPathResult.ORDERED_NODE_SNAPSHOT_TYPE,
    null
  );

  for (let i = 0; i < xPathResult.snapshotLength; i++) {
    xPathDOM.push(xPathResult.snapshotItem(i));
  }

  return xPathDOM;
};

// Custom command overwrite in support/commands.js
//prefix every xpath command with a dot '.' since seeding document.eval
//with a context node.
Cypress.Commands.overwrite("get", (originalFn, selector, options) => {
  let regExpression = /^(\.?\/\/?[*\w])/gim;
  if (typeof selector === "string" && regExpression.test(selector)) {
    cy.document().then(cyDOM => {
      return getElementByXpath(selector, cyDOM.documentElement);
    });
  } else {
    return originalFn(selector, options);
  }
});

[fringd]

can someone maybe give me a hint? where is cy.get defined? I'm struggling to track it down.

[jennifer-shehane]

@fringd cy.get definition within Cypress repo: https://github.com/cypress-io/cypress/blob/develop/packages/driver/src/cy/commands/querying.coffee#L69

[Manishku99251]

Another +1 for Xpath support in the Cypress .get() method

Please add this support as soon as possible

[jennifer-shehane]

@bahmutov has implemented custom xpath commands as a Cypress plugin that some of you may find useful here: https://github.com/cypress-io/cypress-xpath

[scorpyto]

In my opinion .xpath is far more used than .get command. Considering it is just an xpath there should be no changes at all so the support won't be that time consuming? What is bothering me is all the test I have right now -> are they going to stop working like in 2 months or so for example or not?

[jesusiglesias]

It would be great if Cypress natively supports Xpath :)

[HoolioIL]

Unbelievable, we wanted to use Cypress in our company and also buy Cypress Cloud, in these last weeks we have created a demo for this purpose, but we are using xpath because it is easier for us to find elements (and it is easier to migrate the current cases we have), we also have in most cases of the UI reusable components, only the text is different so xpath is a required and essential function for us. If it will not be maintained is a lot the risk that in future versions we have problems, it is a shame...

[scorpyto]

Unbelievable, we wanted to use Cypress in our company and also buy Cypress Cloud, in these last weeks we have created a demo for this purpose, but we are using xpath because it is easier for us to find elements (and it is easier to migrate the current cases we have), we also have in most cases of the UI reusable components, only the text is different so xpath is a required and essential function for us. If it will not be maintained is a lot the risk that in future versions we have problems, it is a shame...

I completely agree! Nowadays developers use more and more dynamic js objects without id or some main attributes and it is impossible to create reliable test cases without xpaths. I guess a lot of people will stop using Cypress if xpath problem starts to appear.

[matbgn]

Incredible, we migrated from TagUI to Cypress for about 3 months, and now we encounter this issue, meaning that we have to migrate all xpath "legacy" call. Sad...

[hansyohan]

will there be any future update on this? I mean will Cypress support this natively?

[atalanchuk-cafemedia]

Please add the XPath support back

[Hellsfoul]

Without xpath, Cypress is not usable for us anymore. In our project, we test a website from a contractor. Good luck to convince them, to add ids everywhere, nevertheless the time it takes until the complex system has them everywhere. CSS is not enough to find the elements needed. Sometimes we just need the //div[contains(text(), "myText")] xpath, to find the correct element.

[bahmutov]

You do have cy.contains command …Sent from my iPhoneOn Aug 9, 2023, at 09:32, Hellsfoul @.***> wrote: Without xpath, Cypress is not usable for us anymore. In our project, we test a website from a contractor. Good luck to convince them, to add ids everywhere, nevertheless the time it takes until the complex system has them everywhere. CSS is not enough to find the elements needed. Sometimes we just need the //div[contains(text(), "myText")] xpath, to find the correct element.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

[mojiAh]

@bahmutov but cy.contains cannot handle xpath or it can? Isn't it easier to bring back cypress-xpath again?

[bahmutov]

No, but you can chain cy.get, cy.contains and find elements by text, no xpath necessary Sent from my iPhoneOn Aug 9, 2023, at 11:37, Moji @.***> wrote: @bahmutov but cy.contains cannot handle xpath or it can? Isn't it easier to bring back cypress-xpath again?

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

[mojiAh]

@bahmutov It is true when I'm starting a project, but in our case transitioning from xpath in our existing tests means we should rewrite pretty much all the tests which it doesn't make any sense(it does make sense if we get more value but simply we don't). Can you explain the reason of deprecating the @cypress/xpath while it did make sense to have it in first place?