Support for fields in Shadow DOM

[ziontee113]

Setup

Browser: Firefox Editor: Neovim

Description

Thank you for the amazing extension. It works pretty much everywhere except for local installation of AUTOMATIC1111 Stable Diffusion WebUI. https://github.com/AUTOMATIC1111/stable-diffusion-webui

But it does work on https://huggingface.co/spaces/camenduru/webui

I think it has something to do with the way GhostText detects textareas inside this <gradio-app> tag, but I'm not sure.

Please tell me which other information I can provide to clarify this issue. Thank you very much.

[fregante]

Indeed it doesn't work when fields are in the Shadow DOM:

• Scroll to the bottom: https://ghosttext-fregante-31vstsqen-fregante.vercel.app/test
• https://github.com/fregante/ghosttext.fregante.com/pull/15

[fregante]

Those fields don't seem to be easily accessible: https://developer.chrome.com/blog/remove-shadow-piercing/

The only way to find them would be to loop through every element on the page searching for shadow roots, like:

$$('*').flatMap(el => [...el.shadowRoot?.querySelectorAll('textarea')??[]])

This piece of code does find the shadowed textarea at the bottom of the Test page, but it might be expensive to run on large pages. It would be interesting nonetheless. PR welcome (ideally supporting other fields)

https://github.com/fregante/GhostText/blob/c33d6bf04089655043c14d3d0fd85c1df96ed49c/source/ghost-text.js#L262

Example:

function registerElements(root) {
    for (const element of root.querySelectorAll(selector)) {
        // TODO: Only handle areas that are visible
        //  && element.getBoundingClientRect().width > 20
        if (!knownElements.has(element)) {
            knownElements.set(element, new GhostTextField(element));
        }
    }
}

registerElements(document);
for (const {shadowRoot} of document.querySelectorAll('*')) {
    if (shadowRoot) {
        registerElements(shadowRoot);
    }
}

[ziontee113]

Thank you for the info. If it's expensive, we could dedicate its own "button". If the user wants to deliberately seek out those shadow elements, they know what they're getting themselves into.

[fregante]

A extension API was recently added for this:

• https://developer.chrome.com/docs/extensions/reference/api/dom

But it would only enable access if GhostText already knows which element has a shadow root, I think. So probably it can only detect fields if you right click on them for example, and only in Firefox since it's the only browser that tells you which element was right clicked:

• https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/menus/getTargetElement

This might eventually be useful if added:

• https://github.com/w3c/webextensions/issues/624