Open marcusdiy opened 11 months ago
Can you share the example from a webpage or component?
Sure. Just create a round button and try to click it.
const GhostCursor = require("ghost-cursor");
const puppeteer = require("puppeteer");
(async () => {
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage()
const cursor = GhostCursor.createCursor(page);
await GhostCursor.installMouseHelper(page);
//await page.goto('https://example.com/');
await page.reload();
await page.evaluate(() => {
let h = `
<div id="missedCount">Missed count: <b>0</b></div>
<div id="btn">Click me</div>
<style> #btn {border-radius: 93% 7% 96% 4% / 93% 6% 94% 7%; padding: 30px; background: #ffdcdc} </style>`;
let s = document.createElement('div'); s.innerHTML = h; document.body.appendChild(s);
});
await page.evaluate(() => {
let h = `
let missed = 0;
document.body.addEventListener('click', () => { missed++; updateCount(); });
document.querySelector('#btn').addEventListener('click', (e) => { e.stopImmediatePropagation(); });
function updateCount() { document.querySelector('#missedCount b').innerText = missed; }`;
let s = document.createElement('script'); s.innerHTML = h; document.body.appendChild(s);
});
while (true) {
let $btn = await page.$('#btn');
await cursor.click($btn).catch(console.error);
}
})();
A solution could be checking if the element hovered is actually our target, and if not retry.
document.elementFromPoint(e.clientX, e.clientY)
also having this issue
same
same issue
Any update on this issue? Facing same challenge
The problem is that CDP returns a rectangle bouncing box. So while clicking in the middle is usually OK, a random point might be outside of the clickable area in a non-rectangle element. For example:
I'm not sure if there is an easy way to fix this. Maybe add an extra check upon calculating a point to see if it's hoverable?
@Niek This is what ChatGPT suggests in a nutshell.. Can this solution be incorporated?
Find border radius property of the button
You can safely click within a rectangle defined by [x + r, y + r] to [x + width - r, y + height - r]. Ensure to handle cases where r might be large enough to affect more than just the corners (e.g., circular buttons)
async function clickRandomPoint(buttonSelector) {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://yourwebsite.com');
const rect = await page.evaluate(selector => {
const element = document.querySelector(selector);
const {x, y, width, height} = element.getBoundingClientRect();
const style = window.getComputedStyle(element);
const radius = parseInt(style.borderRadius); // Assuming uniform radius
return {x, y, width, height, radius};
}, buttonSelector);
// Calculate safe area considering border radius
const safeX = rect.x + rect.radius;
const safeY = rect.y + rect.radius;
const safeWidth = rect.width - 2 * rect.radius;
const safeHeight = rect.height - 2 * rect.radius;
// Generate random coordinates within the safe area
const randomX = safeX + Math.random() * safeWidth;
const randomY = safeY + Math.random() * safeHeight;
// Click the random point
await page.mouse.click(randomX, randomY);
await browser.close();
}
Full prompt + answer here - https://chat.openai.com/share/2be55fbf-1fa1-429f-8125-7c21b6534ff9
That would only work for border-radius styled elements. There are lots of other cases where the clickable object is not a perfect rectangle. I saw it e.g. on SERP links, where links with linebreaks are only clickable on the exact text. Best would be to have a generalized fix for this issue.
One solution is to set paddingPercentage: 99
(or another high value) in your move
/click
function. This will make the cursor always go near the center.
I just want to voice this crutch - you can change the css of the element before clicking so that style="border-radius: 0 "
OR
await page.addStyleTag({
content: `* {
border-radius: 0 !important;
}`
})
When the button is a circle or has rounded box the click may happen to be outside the button. There is needed a better solution to handle rounded shapes otherwise clicks wont work.