search

microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
65.74k stars 3.58k forks source link

[BUG] codegen producing weird selectors #6168

Closed raidneural closed 2 years ago

raidneural commented 3 years ago

Context:

System: OS: macOS 11.2.3 Memory: 1.36 GB / 16.00 GB Binaries: Node: 15.13.0 - /usr/local/bin/node npm: 7.7.6 - /usr/local/bin/npm Languages: Bash: 3.2.57 - /bin/bash

Code Snippet

Help us help you! Put down a short code snippet that illustrates your bug and that we can run and debug locally. For example:


const { chromium } = require('playwright');
(async () => {
  const browser = await chromium.launch({
    headless: false
  });
  const context = await browser.newContext({
    locale: 'en-GB'
  });
  // Open new page
  const page = await context.newPage();
  // Go to https://xxx.my.salesforce.com/
  await page.goto('https://xxx.my.salesforce.com/');
  // Fill input[name="username"]
  await page.fill('input[name="username"]', 'USERNAME_HERE');
  // Press Tab
  await page.press('input[name="username"]', 'Tab');
  // Fill input[name="pw"]
  await page.fill('input[name="pw"]', 'PASSWORD_HERE');
  // Press Enter
  await page.press('input[name="pw"]', 'Enter');
  // Go to https://businez.lightning.force.com/one/one.app
  await page.goto('https://businez.lightning.force.com/one/one.app');
  // Go to https://businez.lightning.force.com/lightning/page/home
  await page.goto('https://businez.lightning.force.com/lightning/page/home');
  // Click text=Switch to Salesforce Classic
  await page.click('text=Switch to Salesforce Classic');
**// AT THIS POINT, I CLICKED ONE OF THE TABS. INSTEAD GRABBING THE TEXT, PLAYWRIGHT USED A GOTO**
  // assert.equal(page.url(), 'https://businez.my.salesforce.com/home/home.jsp?source=lex');
  // Go to https://businez.my.salesforce.com/003/o
  await page.goto('https://businez.my.salesforce.com/003/o');
**// CLICKING IN A DIFFERENT TAB PRODUCES THE EXPECTED SELECTOR...**
  // Click text=Accounts
  await page.click('text=Accounts');
  // assert.equal(page.url(), 'https://businez.my.salesforce.com/001/o');
**// NEXT: MORE WEIRD SELECTORS**
  // Click text=Setup
  await page.click('text=Setup');
  // assert.equal(page.url(), 'https://businez.my.salesforce.com/setup/forcecomHomepage.apexp?setupid=ForceCom&retURL=%2F001%2Fo');
  // Click img[alt="Expand - Manage Users - Level 1"]
  await page.click('img[alt="Expand - Manage Users - Level 1"]');
  // Click text=Permission Sets
  await page.click('text=Permission Sets');
  // assert.equal(page.url(), 'https://businez.my.salesforce.com/0PS?setupid=PermSets&retURL=%2Fui%2Fsetup%2FSetup%3Fsetupid%3DUsers');
  // Click text=Clone
  await page.click('text=Clone');
  // assert.equal(page.url(), 'https://businez.my.salesforce.com/udd/PermissionSet/newPermissionSet.apexp?retURL=%2F0PS%3Ffcf%3D00B5e000001j4oK%26rolodexIndex%3D-1%26page%3D1&id=0PS5e000000wrye&setupid=PermSets');
  // Go to https://businez.my.salesforce.com/0PS?setupid=PermSets&retURL=%2Fui%2Fsetup%2FSetup%3Fsetupid%3DUsers
  await page.goto('https://businez.my.salesforce.com/0PS?setupid=PermSets&retURL=%2Fui%2Fsetup%2FSetup%3Fsetupid%3DUsers');
  // Click text=On this page you can create, view, and manage permission sets. In addition, you
  await page.click('text=On this page you can create, view, and manage permission sets. In addition, you ');
**// THIS IS THE EXPECTED SELECTOR, OR A SIMILAR ONE, WHEN CLICKING 'CLONE' LINKS**
  // Click text=Clone
  await page.click('text=Clone');
  // assert.equal(page.url(), 'https://businez.my.salesforce.com/udd/PermissionSet/newPermissionSet.apexp?retURL=%2F0PS%3Ffcf%3D00B5e000001j4oK%26rolodexIndex%3D-1%26page%3D1&id=0PS5e000000wrye&setupid=PermSets');
  // Go to https://businez.my.salesforce.com/0PS?setupid=PermSets&retURL=%2Fui%2Fsetup%2FSetup%3Fsetupid%3DUsers
  await page.goto('https://businez.my.salesforce.com/0PS?setupid=PermSets&retURL=%2Fui%2Fsetup%2FSetup%3Fsetupid%3DUsers');
**// THESE ARE THE SELECTORS GENERATED FOR THE OTHER 'CLONE' LINKS**
  // Click #ext-gen11 div:has-text("0PS5e000000wrydCloneSalesforce Console UserEnable Salesforce Console UserSales C")
  await page.click('#ext-gen11 div:has-text("0PS5e000000wrydCloneSalesforce Console UserEnable Salesforce Console UserSales C")');
  // Click #ext-gen11 div:has-text("0PS5e000000wrydCloneSalesforce Console UserEnable Salesforce Console UserSales C")
  await page.click('#ext-gen11 div:has-text("0PS5e000000wrydCloneSalesforce Console UserEnable Salesforce Console UserSales C")');
  // Click #ext-gen11 div:has-text("0PS5e000000wrycCloneSalesforce CMS Integration AdminGives the admin data access ")
  await page.click('#ext-gen11 div:has-text("0PS5e000000wrycCloneSalesforce CMS Integration AdminGives the admin data access ")');
  // Click #ext-gen11 div:has-text("0PS5e0000001sXqCloneStandard Einstein Activity CaptureAccess to Standard Einstei")
  await page.click('#ext-gen11 div:has-text("0PS5e0000001sXqCloneStandard Einstein Activity CaptureAccess to Standard Einstei")');
  // ---------------------
  await context.close();
  await browser.close();
})();

Describe the bug

Hi, thanks for all the work you do on this project - its awesome

I'm finding different problems when generating selectors in the app above in different cases:

  1. The case with the tabs commented in the code above - it's like when the page it's initially loaded, playwright behaves different. After clicking a second tab, the selectors produced where the expected, i.e. using the 'Text' engine
  2. The different behaviors when clicking the different 'Clone' links under the 'Setup' page. I expected different selectors, since there are several links with that text, but not the shown above
  3. Finally, sometimes I click a simple link with a clear text, like 'Setup', and instead of getting a nice Text selector, I got a page.goto()
+18ms pw:api attempting click action +2ms pw:api waiting for element to be visible, enabled and stable +0ms pw:api element is visible, enabled and stable +10ms pw:api scrolling into view if needed +0ms pw:api done scrolling +0ms pw:api checking that element receives pointer events at (734,373) +1ms pw:api element does receive pointer events +3ms pw:api performing click action +0ms pw:api click action done +58ms pw:api waiting for scheduled navigations to finish +0ms pw:api navigations have finished +3ms

Again, thank you for the awesome project!

dgozman commented 3 years ago

Hi, thanks for all the work you do on this project - its awesome

Thank you for the kind words!

  1. The case with the tabs commented in the code above - it's like when the page it's initially loaded, playwright behaves different. After clicking a second tab, the selectors produced where the expected, i.e. using the 'Text' engine

The snippet I see has a pretty nice text=Setup selector. Are you talking about a different one?

**// NEXT: MORE WEIRD SELECTORS**
  // Click text=Setup
  1. The different behaviors when clicking the different 'Clone' links under the 'Setup' page. I expected different selectors, since there are several links with that text, but not the shown above

Just a guess: text "Clone" appears on the page multiple times, so codegen uses a different selector to ensure we target the correct element. You can try evaluating playwright.$$('text=Clone') in DevTools console while recording to check how many elements match the selector you think is best.

  1. Finally, sometimes I click a simple link with a clear text, like 'Setup', and instead of getting a nice Text selector, I got a page.goto()

This sounds like a bug! Unfortunately, I do not have a Salesforce account, so reproducing this is very hard. If you could find a publicly available repro case, we would be able to debug. Any chance this happens on some other pages as well? If you have a repro that you are comfortable sharing privately, send me something to dgozman [at] microsoft [dot] com.

raidneural commented 3 years ago

@dgozman thanks for your reply

Let me further explain my thoughs on the above:

The two lines refer to a tab, but the first case used a page.goto with the destination url instead of clicking the tab using the text as reference, just like with the Accounts tab

I'll reach you via email in order to share some credentials with you, plus further details on the different findings

Thank you very much for your time!

raidneural commented 3 years ago

@dgozman I just sent out some emails to you pointing out how to proceed to test the above by yourself

Thanks

dgozman commented 3 years ago

I was able to reproduce the "goto instead of click" bug. I think this is because previous click did not finish yet, and so we are not recording the next click when it's done too fast, and instead get the goto. Let me see what we can do here.

raidneural commented 3 years ago

@dgozman I sent out some information about the other issue with the "Clone" links. Feel free to check your inbox and come back to me if you need further details - hope you will be able to reproduce it with the new info

Thanks for everything! You rock!

raidneural commented 3 years ago

@dgozman I'm kindly following-up on this - did you have a chance to confirm the "Clone" link issue?

Seems like this issue is popping around different places on the website

Thanks++

pavelfeldman commented 2 years ago

Why was this issue closed?

We are prioritizing the bugs based on the upvotes, recency and our ability to act. It looks like this issue only has a handful of upvotes, has not been touched recently and/or we lack sufficient feedback to act on it. We are closing issues like this one to keep our bug database maintainable. Please feel free to open a new issue and link this one to it if you think this is a mistake.