NoDriver: iframe usage

[boludoz]

I'm having difficulty selecting an iframe and using click inside it, I'm using the find and click functions but they don't work inside the iframe. How do I select it?

I really think that nodriver is worth it and I am not going to get off the boat because of this simple obstacle.

My question is, how can I access an iframe within a website?

[boludoz]

from selenium_driverless import webdriver
from selenium_driverless.types.by import By
import asyncio

async def main():
    options = webdriver.ChromeOptions()
    async with webdriver.Chrome(options=options) as driver:
        await driver.get('file:///C:/Users/Admin/Downloads/test_otp.mhtml')
        try:
            iframe = await driver.find_element(By.XPATH, "//iframe[contains(@*, 'OTP')]")
            print("STEP 1 -----------------------------------------------------------------")
            iframe_document = await iframe.content_document
            print("STEP 2 -----------------------------------------------------------------")
            elem = await iframe_document.find_element(By.XPATH, "//*[@id='wcri-html']/body/div[2]/div[1]/div/div/div/fieldset/div/input[1]")
            print("END    -----------------------------------------------------------------")

        except Exception as e:
            print(f"[ERROR] While finding OTP: {e}")

asyncio.run(main())

I liked NoDriver so much that I based it on selenium_driverless, reverse engineering 'selenium_driverless' to try to implement iframe as if you were opening a new tab, I'm trying to reverse engineer it.

[boludoz]

The problem seems to be within find_element_by text Unfortunately it's too much for me.

[boludoz]

[boludoz]

[boludoz]

Fixed:

    async def find_element_by_text(
        self,
        text: str,
        best_match: Optional[bool] = False,
        return_enclosing_element: Optional[bool] = True,
    ) -> Union[element.Element, None]:
        """
        finds and returns the first element containing <text>, or best match

        :param text:
        :type text:
        :param best_match:  when True, which is MUCH more expensive (thus much slower),
                            will find the closest match based on length.
                            this could help tremendously, when for example you search for "login", you'd probably want the login button element,
                            and not thousands of scripts,meta,headings containing a string of "login".

        :type best_match: bool
        :param return_enclosing_element:
        :type return_enclosing_element:
        :return:
        :rtype:
        """
        doc = await self.send(cdp.dom.get_document(-1, True))
        search_id, nresult = await self.send(cdp.dom.perform_search(text, True))
        text = text.strip()

        node_ids = await self.send(cdp.dom.get_search_results(search_id, 0, nresult))
        await self.send(cdp.dom.discard_search_results(search_id))

        if not node_ids:
            node_ids = []
        items = []
        for nid in node_ids:
            node = util.filter_recurse(doc, lambda n: n.node_id == nid)
            try:
                elem = element.create(node, self, doc)
            except:  # noqa
                continue
            if elem.node_type == 3:
                # if found element is a text node (which is plain text, and useless for our purpose),
                # we return the parent element of the node (which is often a tag which can have text between their
                # opening and closing tags (that is most tags, except for example "img" and "video", "br")

                if not elem.parent:
                    # check if parent actually has a parent and update it to be absolutely sure
                    await elem.update()

                items.append(
                    elem.parent or elem
                )  # when it really has no parent, use the text node itself
                continue
            else:
                # just add the element itself
                items.append(elem)

        # since we already fetched the entire doc, including shadow and frames
        # let's also search through the iframes
        iframes = util.filter_recurse_all(doc, lambda node: node.node_name == "IFRAME")
        if iframes:
            iframes_elems = [
                element.create(iframe, self, iframe.content_document)
                for iframe in iframes
            ]
            for nid in node_ids:
                for iframe_elem in iframes_elems:
                    iframe_text_nodes = util.filter_recurse_all(
                        iframe_elem,
                        lambda n: n.node_id == nid)
                    if iframe_text_nodes:
                        iframe_text_elems = [
                            element.create(text_node, self, iframe_elem.tree)
                            for text_node in iframe_text_nodes
                        ]
                        items.extend(text_node.parent for text_node in iframe_text_elems)
        try:
            if not items:
                return
            if best_match:
                closest_by_length = min(
                    items, key=lambda el: abs(len(text) - len(el.text_all))
                )
                elem = closest_by_length or items[0]

                return elem
            else:
                # naively just return the first result
                for elem in items:
                    if elem:
                        return elem
        finally:
            await self.send(cdp.dom.disable())

[boludoz]

The problem now is that I can't click on it.

[WindBora]

@boludoz Can you provide document example that you tried to work on and source code that does not work?

[boludoz]

@boludoz Can you provide document example that you tried to work on and source code that does not work?

<!DOCTYPE html>
<html lang="es">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
</head>
<body>
    <iframe src="https://apoia.se/" 
            width="800" height="600" 
            frameborder="0" scrolling="yes">
    </iframe>
</body>
</html>

[WindBora]

@boludoz

So, after some researching and tweaking I found that we can use iframe as an explicit tab as it is added as a target to driver. That means we can get from driver targets needed for us tab (iframe in our case). That can be done by filter via ID (target_id for target and frame_id for iframe) since they are strings and same for frame and frame tab.

After we get required tab we got to update websocket url since nodriver currently maps it wrongly for some types (like iframe).

Once all that is done, we can use our tab as usual.

Minimal example:

import logging

import nodriver as uc

logging.basicConfig(level=logging.DEBUG)

path_to_website = (
    r"your path to website"
)

async def main():
    driver = await uc.start()

    tab = await driver.get(path_to_website)
    await tab

    # Have to wait until iframe loaded in the system as a target
    await tab.sleep(2)
    # Get required iframe tag in dom structure. From that we will use frame_id
    iframe = await tab.select("iframe")
    # Get required tab. Not safe in case when tab not found
    iframe_tab: uc.Tab = next(
        filter(
            lambda x: str(x.target.target_id) == str(iframe.frame_id), driver.targets
        )
    )
    # Fixing websocket url
    iframe_tab.websocket_url = iframe_tab.websocket_url.replace("iframe", "page")
    button = await iframe_tab.select(
        "#__next > div > div > div.plasmic_apoiase_home_all__EEZNz.PlasmicHome_sectionHero__4yCHj > div > div > div.plasmic_apoiase_home_all__EEZNz.PlasmicHeroHeadline_column__fdMx7__IfB4x > div > div > a"
    )
    await button.click()

    await tab.sleep(1000)

if __name__ == "__main__":
    uc.loop().run_until_complete(main())

[boludoz]

@boludoz

So, after some researching and tweaking I found that we can use iframe as an explicit tab as it is added as a target to driver. That means we can get from driver targets needed for us tab (iframe in our case). That can be done by filter via ID (target_id for target and frame_id for iframe) since they are strings and same for frame and frame tab.

After we get required tab we got to update websocket url since nodriver currently maps it wrongly for some types (like iframe).

Once all that is done, we can use our tab as usual.

Minimal example:

import logging

import nodriver as uc

logging.basicConfig(level=logging.DEBUG)

path_to_website = (
    r"your path to website"
)

async def main():
    driver = await uc.start()

    tab = await driver.get(path_to_website)
    await tab

    # Have to wait until iframe loaded in the system as a target
    await tab.sleep(2)
    # Get required iframe tag in dom structure. From that we will use frame_id
    iframe = await tab.select("iframe")
    # Get required tab. Not safe in case when tab not found
    iframe_tab: uc.Tab = next(
        filter(
            lambda x: str(x.target.target_id) == str(iframe.frame_id), driver.targets
        )
    )
    # Fixing websocket url
    iframe_tab.websocket_url = iframe_tab.websocket_url.replace("iframe", "page")
    button = await iframe_tab.select(
        "#__next > div > div > div.plasmic_apoiase_home_all__EEZNz.PlasmicHome_sectionHero__4yCHj > div > div > div.plasmic_apoiase_home_all__EEZNz.PlasmicHeroHeadline_column__fdMx7__IfB4x > div > div > a"
    )
    await button.click()

    await tab.sleep(1000)

if __name__ == "__main__":
    uc.loop().run_until_complete(main())

I love you so much friend, you really are a genius.

[WindBora]

@boludoz If it solved your problem, consider closing this issue.

[boludoz]

@boludoz If it solved your problem, consider closing this issue.

In that laboratory example it works, but on a real page it doesn't. It must be hidden in the dom something like that. I think the problem is shadow-root

[WindBora]

@boludoz

If you could elaborate more on this problem by providing non-working example, it would be easier to help. If u cannot provide such an example, I don't think I will be of help here

[boludoz]

@boludoz

If you could elaborate more on this problem by providing non-working example, it would be easier to help. If u cannot provide such an example, I don't think I will be of help here

I didn't find shadow-root here, but when I try to find the iframe it doesn't find it. In the example of testing with the HTML it works perfectly, it is as if they were different types of iframe.

I will try to continue investigating and I will inform you of any news. Apparently the 'iframe' is hidden, i changed the filter for a for to debug and it is not able to find the iframe.

[ultrafunkamsterdam]

This would probably help a lot : let me know when you have working code 👍

from nodriver import *
import json

async def main():
    browser = await start()
    tab = await browser.get('https://nowsecure.nl/iframes.html')
    await tab
    frame_data = await tab.send(cdp.page.get_frame_tree())

    print(json.dumps(  frame_data.to_json() , indent=4))

if __name__ == '__main__':
    loop().run_until_complete(main())

result:

{
    "frame": {
        "id": "668AE9308C450E1DC6BAA7F012B48A54",
        "loaderId": "0B5C430EEB56D85056D637C7B464F014",
        "url": "https://nowsecure.nl/iframes.html",
        "domainAndRegistry": "nowsecure.nl",
        "securityOrigin": "https://nowsecure.nl",
        "mimeType": "text/html",
        "secureContextType": "Secure",
        "crossOriginIsolatedContextType": "NotIsolated",
        "gatedAPIFeatures": [],
        "adFrameStatus": {
            "adFrameType": "none"
        }
    },
    "childFrames": [
        {
            "frame": {
                "id": "2B3B5340E99DEB0029848C38D13CC4EF",
                "loaderId": "B2859CDD24B827E4039DB6407F025A62",
                "url": "https://nowsecure.nl/mouse.html",
                "domainAndRegistry": "nowsecure.nl",
                "securityOrigin": "https://nowsecure.nl",
                "mimeType": "text/html",
                "secureContextType": "Secure",
                "crossOriginIsolatedContextType": "NotIsolated",
                "gatedAPIFeatures": [],
                "parentId": "668AE9308C450E1DC6BAA7F012B48A54",
                "name": "iframe0",
                "adFrameStatus": {
                    "adFrameType": "none",
                    "explanations": []
                }
            },
            "childFrames": [
                {
                    "frame": {
                        "id": "26EF37AEC1E7F443A3A9E53AA526DD0D",
                        "loaderId": "401F6DA4EC626470E1DE3DCCEDE09436",
                        "url": "about:blank",
                        "domainAndRegistry": "",
                        "securityOrigin": "://",
                        "mimeType": "text/html",
                        "secureContextType": "Secure",
                        "crossOriginIsolatedContextType": "NotIsolated",
                        "gatedAPIFeatures": [],
                        "parentId": "2B3B5340E99DEB0029848C38D13CC4EF",
                        "name": "",
                        "adFrameStatus": {
                            "adFrameType": "none",
                            "explanations": []
                        }
                    }
                }
            ]
        },
        {
            "frame": {
                "id": "3A0C0B06764092B1DB7F28C202EBC790",
                "loaderId": "FF4552B820C9B8A5AF3903350FEEB0D8",
                "url": "blob:https://nowsecure.nl/99ac4434-cb74-486d-bb6e-413df1273c39",
                "domainAndRegistry": "",
                "securityOrigin": "https://nowsecure.nl",
                "mimeType": "text/html",
                "secureContextType": "Secure",
                "crossOriginIsolatedContextType": "NotIsolated",
                "gatedAPIFeatures": [],
                "parentId": "668AE9308C450E1DC6BAA7F012B48A54",
                "name": "iframe1",
                "adFrameStatus": {
                    "adFrameType": "none",
                    "explanations": []
                }
            },
            "childFrames": [
                {
                    "frame": {
                        "id": "1A18EBDF6588FCD48B226FC7C7568746",
                        "loaderId": "7B58DD268E7BB2B44D2AD77A572487A3",
                        "url": "about:srcdoc",
                        "domainAndRegistry": "",
                        "securityOrigin": "://",
                        "mimeType": "text/html",
                        "secureContextType": "Secure",
                        "crossOriginIsolatedContextType": "NotIsolated",
                        "gatedAPIFeatures": [],
                        "parentId": "3A0C0B06764092B1DB7F28C202EBC790",
                        "name": "iframe-in-iframe",
                        "adFrameStatus": {
                            "adFrameType": "none",
                            "explanations": []
                        }
                    }
                }
            ]
        },
        {
            "frame": {
                "id": "10CAF055964D3655CD6E4B53958F1ADB",
                "loaderId": "8E15344567C5A95855A43725FDB9B01F",
                "url": "about:blank",
                "domainAndRegistry": "",
                "securityOrigin": "://",
                "mimeType": "text/html",
                "secureContextType": "Secure",
                "crossOriginIsolatedContextType": "NotIsolated",
                "gatedAPIFeatures": [],
                "parentId": "668AE9308C450E1DC6BAA7F012B48A54",
                "name": "",
                "adFrameStatus": {
                    "adFrameType": "none",
                    "explanations": []
                }
            }
        }
    ]
}

Also

>>> from nodriver import *
>>> browser = await start()
>>> tab = await browser.get('https://nowsecure.nl/iframes.html')
>>> await tab
>>> iframes = await tab.select_all('iframe')
>>> boxes = await iframes[0].query_selector_all('.box')
>>> for box in boxes:
        await box.scroll_into_view()
        await box.mouse_move()
        await box.flash()
        await tab.wait(1)

https://github.com/ultrafunkamsterdam/undetected-chromedriver/assets/21027969/fd02e8e9-d9df-4309-9fcc-47966189e0fc

[WindBora]

@ultrafunkamsterdam

Regarding your code. It will probably not work in real life as you use iframe with same origin sources. If you will use from different origin then it will not work. An example of the file was provided by @boludoz . (Cross site protection - something like that this is called)

@boludoz Can you provide document example that you tried to work on and source code that does not work?

<!DOCTYPE html>
<html lang="es">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
</head>
<body>
    <iframe src="https://apoia.se/" 
            width="800" height="600" 
            frameborder="0" scrolling="yes">
    </iframe>
</body>
</html>

[WindBora]

@boludoz

Does your website (razer) with its OTP uses same origin for iframe? Like source of iframe is same as the website you visit? If yes, then answer of @ultrafunkamsterdam will probably solve your problems

[ultrafunkamsterdam]

untested

config = uc.Config()
config.add_argument('--disable-web-security')
driver = await uc.start(config)

iirc it will tell you 'a unsupported option is used', but will work cross origin iframes

[boludoz]

untested

config = uc.Config()
config.add_argument('--disable-web-security')
driver = await uc.start(config)

iirc it will tell you 'a unsupported option is used', but will work cross origin iframes

Do you have an email to send you information?

[boludoz]

Iframe control (1).pdf I’ve shared some commands that I’ve managed to reverse engineer and they’ve been successful. I haven’t yet turned them into code, but I’ve tried to provide as detailed a description as possible.

The funniest thing is that I managed to navigate through the frames by modifying get and navigate but I couldn't get what they have inside.

[boludoz]

I made a little progress, I'm just having a hard time running the javascripts.

import asyncio
import nodriver as uc
from nodriver import *

import asyncio

async def main():
        config = uc.Config()
        config.add_argument('--disable-web-security')
        browser = await uc.start(config)
        tab = await browser.get('file:///C:/Users/franc/Downloads/The%20New%20Razer%20Gold%20&%20Silver.mhtml')

        iframes = await tab.select_all('iframe')
        iframe = iframes[0]
        print(iframe.backend_node_id)
        print(iframe.frame_id)
        print("IFRAMES -------------------")
        isolated_world_context = await tab.send(cdp.page.create_isolated_world(frame_id=iframe.frame_id, world_name='isolated', grant_univeral_access=True))
        print(isolated_world_context)
        print("isolated_world_context -------------------")
        resolve_node_id = await tab.send(cdp.dom.resolve_node(node_id=iframe.backend_node_id, execution_context_id=isolated_world_context, object_group='isolated'))
        print(resolve_node_id.object_id)
        print("resolve_node_id -------------------")

        arguments = [cdp.runtime.CallArgument(object_id=resolve_node_id.object_id)]
        result_js = await tab.send(
            cdp.runtime.call_function_on(
                "() => { return document.body.innerHTML; }",
                object_id=resolve_node_id.object_id,
                arguments=arguments,
                await_promise=True,
                user_gesture=True,
                return_by_value=True,
            )
        )

        print(result_js)

        print("call_function_on_result -------------------")
        exit()
asyncio.run(main())

[WindBora]

At this point, without any real example of the website I am unable to help you...

[boludoz]

At this point, without any real example of the website I am unable to help you...

Can you invite me with your email without the at sign?

[ultrafunkamsterdam]

I made a little progress, I'm just having a hard time running the javascripts.

import asyncio
import nodriver as uc
from nodriver import *

import asyncio

async def main():
        config = uc.Config()
        config.add_argument('--disable-web-security')
        browser = await uc.start(config)
        tab = await browser.get('file:///C:/Users/franc/Downloads/The%20New%20Razer%20Gold%20&%20Silver.mhtml')

        iframes = await tab.select_all('iframe')
        iframe = iframes[0]
        print(iframe.backend_node_id)
        print(iframe.frame_id)
        print("IFRAMES -------------------")
        isolated_world_context = await tab.send(cdp.page.create_isolated_world(frame_id=iframe.frame_id, world_name='isolated', grant_univeral_access=True))
        print(isolated_world_context)
        print("isolated_world_context -------------------")
        resolve_node_id = await tab.send(cdp.dom.resolve_node(node_id=iframe.backend_node_id, execution_context_id=isolated_world_context, object_group='isolated'))
        print(resolve_node_id.object_id)
        print("resolve_node_id -------------------")

        arguments = [cdp.runtime.CallArgument(object_id=resolve_node_id.object_id)]
        result_js = await tab.send(
            cdp.runtime.call_function_on(
                "() => { return document.body.innerHTML; }",
                object_id=resolve_node_id.object_id,
                arguments=arguments,
                await_promise=True,
                user_gesture=True,
                return_by_value=True,
            )
        )

        print(result_js)

        print("call_function_on_result -------------------")
        exit()
asyncio.run(main())

This is indeed quirky. The iframe document (not the tag) should be passed..

myframe.content_document.node_id

[boludoz]

I made a little progress, I'm just having a hard time running the javascripts.

import asyncio
import nodriver as uc
from nodriver import *

import asyncio

async def main():
        config = uc.Config()
        config.add_argument('--disable-web-security')
        browser = await uc.start(config)
        tab = await browser.get('file:///C:/Users/franc/Downloads/The%20New%20Razer%20Gold%20&%20Silver.mhtml')

        iframes = await tab.select_all('iframe')
        iframe = iframes[0]
        print(iframe.backend_node_id)
        print(iframe.frame_id)
        print("IFRAMES -------------------")
        isolated_world_context = await tab.send(cdp.page.create_isolated_world(frame_id=iframe.frame_id, world_name='isolated', grant_univeral_access=True))
        print(isolated_world_context)
        print("isolated_world_context -------------------")
        resolve_node_id = await tab.send(cdp.dom.resolve_node(node_id=iframe.backend_node_id, execution_context_id=isolated_world_context, object_group='isolated'))
        print(resolve_node_id.object_id)
        print("resolve_node_id -------------------")

        arguments = [cdp.runtime.CallArgument(object_id=resolve_node_id.object_id)]
        result_js = await tab.send(
            cdp.runtime.call_function_on(
                "() => { return document.body.innerHTML; }",
                object_id=resolve_node_id.object_id,
                arguments=arguments,
                await_promise=True,
                user_gesture=True,
                return_by_value=True,
            )
        )

        print(result_js)

        print("call_function_on_result -------------------")
        exit()
asyncio.run(main())

This is indeed quirky. The iframe document (not the tag) should be passed..

myframe.content_document.node_id

I'm sorry, it looks like that code had errors. I have made even greater progress, the only problem I have is that apparently my iframe is not detected with conventional methods.

[boludoz]

That has worked perfectly, the other thing I would like is a high-level function to execute cdp/javascript commands, like the one selenium has. I will apreciate a lot.

[boludoz]

import asyncio
import nodriver as uc
from nodriver import *

import asyncio

async def main():
        config = uc.Config()
        config.add_argument('--disable-web-security')
        browser = await uc.start(config)
        tab = await browser.get('file:///C:/Users/franc/Downloads/The%20New%20Razer%20Gold%20&%20Silver.mhtml')

        iframes = await tab.select_all('iframe')
        iframe = iframes[0]
        print(iframe.backend_node_id)
        print(iframe.frame_id)
        print("IFRAMES -------------------")
        isolated_world_context = await tab.send(cdp.page.create_isolated_world(frame_id=iframe.frame_id, world_name='isolated', grant_univeral_access=True))
        print(isolated_world_context)
        print("isolated_world_context -------------------")

        iframe.content_document.node_id = await tab.send(cdp.dom.request_child_nodes(node_id=iframe.content_document.node_id, depth=-1))
        print(iframe.content_document.node_id)
        print("iframe.content_document.node_id -------------------")

        resolve_node_id = await tab.send(cdp.dom.resolve_node(node_id=iframe.content_document.node_id, execution_context_id=isolated_world_context, object_group='isolated'))
        print(resolve_node_id.object_id)
        print("resolve_node_id -------------------")

        arguments = [cdp.runtime.CallArgument(object_id=resolve_node_id.object_id)]
        result_js = await tab.send(
            cdp.runtime.call_function_on(
            "() => { return document.body.innerHTML; }",
            object_id=resolve_node_id.object_id,
            arguments=arguments,
            await_promise=True,
            user_gesture=True,
            return_by_value=True,
            )
        )

        print(result_js)

        print("call_function_on_result -------------------")

        arguments = [cdp.runtime.CallArgument(value='1')]  # El valor que quieres ingresar
        result_js = await tab.send(
            cdp.runtime.call_function_on(
            """
            (value) => { 
                // Asegúrate de que este XPath corresponda al campo de entrada que quieres llenar
                let xpath = '//*[@id="wcri-html"]/body/div[2]/div[1]/div/div/div/fieldset/div/input[1]';
                let inputElement = document.evaluate(xpath, document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue;
                inputElement.value = value;
            }
            """,
            object_id=resolve_node_id.object_id,
            arguments=arguments,
            await_promise=True,
            user_gesture=True,
            return_by_value=True,
            )
        )

        exit()

asyncio.run(main())

The following code works for me, but I'm still struggling to get a more systematic way to interact.

The problem seems to be that the iframe has the following characteristic: CrossOriginIsolatedContextType.NOT_ISOLATED_FEATURE_DISABLED

[ultrafunkamsterdam]

use --disable-web-security as option . it will disable all isolations and enable to interact directly . However it will show an warning under the url bar stating that it is an unsupported option (but it is supported).

If if recall correctly, this even enables find() and select() methods to work, but it's such a niche what your're doing, and not something you come acrross often in regular use cases or sites.

[boludoz]

use --disable-web-security as option . it will disable all isolations and enable to interact directly . However it will show an warning under the url bar stating that it is an unsupported option (but it is supported).

If if recall correctly, this even enables find() and select() methods to work, but it's such a niche what your're doing, and not something you come acrross often in regular use cases or sites.

Unfortunately this option has not worked for me and what seems niche in reality according to my experience becomes custom, it seems that there is an error in the filter of your find function (you must add an additional for for each nid), I have made a custom one and it worked, I have shared it above and it has some shortcomings with normal frames. Doing this without using a selector is 100% possible, although I would like to find some way to make it also visible like the rest of the frames, although it seems much more practical to use find.

I admit that I am very angry with the person who designed the iframe.

[LinuxTux193]

can you please share the code? i can't select the payment iframe on the nike website