Custom data attribute not working or misunderstand its usage?

[ethantran]

Reproducible steps:

1. Input data-ved or ved in settings of extension

   

2. Go to "google.com"

3. Press "Google Search" button

4. Received code

   const puppeteer = require('puppeteer');
   (async () => {
   const browser = await puppeteer.launch()
   const page = await browser.newPage()
   
   await page.goto('https://www.google.com/')
   
   await page.setViewport({ width: 983, height: 685 })
   
   await page.waitForSelector('div > .A8SBwf > .FPdoLc > center > .gNO89b')
   await page.click('div > .A8SBwf > .FPdoLc > center > .gNO89b')
   
   await browser.close()
   })()

5. Expected

   const puppeteer = require('puppeteer');
   (async () => {
   const browser = await puppeteer.launch()
   const page = await browser.newPage()
   
   await page.goto('https://www.google.com/')
   
   await page.setViewport({ width: 983, height: 685 })
   
   await page.waitForSelector('[data-ved="0ahUKEwjG8ZGz-MvkAhXRqZ4KHb3SClEQ4dUDCAc"]')
   await page.click('[data-ved="0ahUKEwjG8ZGz-MvkAhXRqZ4KHb3SClEQ4dUDCAc"]')
   
   await browser.close()
   })()

[srhise]

This is also not working for me.

data attribute data-ref also tried just ref

HTML

<a href="/get-started" className="primary-btn w-button" data-ref="1">
   Get Started
</a>

Received:

const puppeteer = require('puppeteer');
(async () => {
  const browser = await puppeteer.launch()
  const page = await browser.newPage()

  const navigationPromise = page.waitForNavigation()

  await page.goto('http://localhost:3000/')

  await page.setViewport({ width: 1658, height: 1228 })

  await page.waitForSelector('.jsx-3593952768 > .jsx-3593952768:nth-child(1) > .jsx-3593952768 > .jsx-3593952768 > .jsx-3593952768')
  await page.click('.jsx-3593952768 > .jsx-3593952768:nth-child(1) > .jsx-3593952768 > .jsx-3593952768 > .jsx-3593952768')

  await navigationPromise

  await browser.close()
})()

[tnolet]

@ethantran @srhise can one of you give me a site / example where this is reproducable? Google is really finicky

[ethantran]

@tnolet I have tried several websites with a data attribute. Google search button has a data-ved attribute. The New York Times has a data-testid for their nav button on top left corner. Can you show us how to use that setting on The New York Times or Google?

[ycjcl868]

+1

[zhandzlik]

@tnolet Is there any update to this issue? I'm experiencing the same thing on a local site as well as google.com and The New York Times as @ethantran mentioned above

[tnolet]

@zhandzlik sorry no update that's going to help you out here, only that this project is not forgotten and will receive a lot of love and attention ASAP once I can free up some time. PR's are 100% welcome to fix any bugs.

[a631807682]

It work for me when i setting data-ved with google.

const puppeteer = require('puppeteer');
(async () => {
  const browser = await puppeteer.launch()
  const page = await browser.newPage()

  const navigationPromise = page.waitForNavigation()

  await page.goto('https://www.google.com/')

  await page.setViewport({ width: 1280, height: 610 })

  await navigationPromise

  await page.waitForSelector('div > .A8SBwf > .FPdoLc > center > [data-ved="0ahUKEwjwvrz-s4XoAhVDI6YKHbotAowQ4dUDCAo"]')
  await page.click('div > .A8SBwf > .FPdoLc > center > [data-ved="0ahUKEwjwvrz-s4XoAhVDI6YKHbotAowQ4dUDCAo"]')

  await browser.close()
})()

I don't think it's a bug, but the data-* of dataAttribute description is hard to understand. In fact,it works only if we set the attribute exactly equal to those of the DOM. https://github.com/checkly/puppeteer-recorder/blob/master/src/content-scripts/EventRecorder.js#L105

[prlyons]

I tried data-ved and google.com with no luck. I used Chrome and Chrome beta. Our web application generates id dynamically so we rely on the custom data attribute. Thanks!

[tnolet]

@prlyons I cannot reproduce this behavior. Setting data-ved yields the correct selector. As per the original issue submitter @ethantran I do:

1. I set data-ved as custom data attribute.
2. Go to "google.com"
3. Press "Google Search" button
4. Received code

const puppeteer = require('puppeteer');
(async () => {
  const browser = await puppeteer.launch()
  const page = await browser.newPage()

  await page.goto('https://www.google.com/')

  await page.setViewport({ width: 2548, height: 867 })

  await page.waitForSelector('div > .A8SBwf > .FPdoLc > center > [data-ved="0ahUKEwjY5-zk8qvqAhWBC-wKHf0WCXIQ4dUDCA0"]')
  await page.click('div > .A8SBwf > .FPdoLc > center > [data-ved="0ahUKEwjY5-zk8qvqAhWBC-wKHf0WCXIQ4dUDCA0"]')

  await browser.close()
})()

[ianaya89]

I think that this could be related with elements that have id and data-* attributes. Will be reviewing this and get back with a fix. In the meantime, if anyone that faced this problem before could help to confirm would be great.

[ianaya89]

I added some changes that will improve this a little bit, but we can't consider it 100% fixed. (https://github.com/checkly/headless-recorder/pull/130)

• If you set data-* and your target element has data-* it will take precedence and selector output will be [data-*="the value"].
• if the data-* is not directly in the target element but it is present in its parents/siblings this could still has issues depending of the presence of the id attribute.

Basically when working directly when target element, data-* takes priority over id but when data-* is not directly in the target and it is in parents/siblings we can't give priority to data-* and the final output will be based on id.

The problem we have is limitation with the library we use to get the unique css selector for each element. For the scenario mentioned above, we are not able to give priority to other attribute rather than id. As I said, we can not consider this fixed, but will add better support for most of scenarios.

We are also working on a new engine to genere our own css selectors and make it smarter. That new feature is still in progress