Error Chrome Extension for testing added at the test runner level

[eduardoacskimlinks]

Current behavior

The extension is added in the context of Chrome browser instead of the iframe context causing problems in running the extension tests like

• Incorrect URL, the extension gets the ...__specs URL instead of the nike.com URL as it should
• Extension component added outside of the IFRAME testing context.

Desired behavior

The extension is loaded within the testing context so we can perform end-to-end tests effectively.

Test code to reproduce

You can see this problem with any extension; if you want to use ours, feel free to download it from

https://chrome.google.com/webstore/detail/skimlinks-editor-tool/gmfcchdbgideoiehadmlmpnbhdcjoaih?hl=en-GB

Alternatively, the same issue is presently using https://github.com/leonbubova/cypress-chrome-extension-minimal-reproduction, although it doesn't read URL data on the background script.

Cypress Version

12.5.1

Node version

v16.16.0

Operating System

macOS 11.4

Debug Logs

No response

Other

No response

[marktnoonan]

Hi @eduardoacskimlinks, thanks for reporting this. I don't know much about testing chrome extensions but hunting around I saw a suggestion to use the all_frames option in your extension's manifest to avoid the extension only being added to window.top.

It's a guess on my part, but that seems like it would explain why your extension injects itself and is getting the location of window.top instead of the iframe.

Please try this out and let me know if it addresses your problem.

[eduardoacskimlinks]

@marktnoonan, thanks for reaching out. I have tried all_frames, but it doesn't solve the underlying issue for testing chrome extensions on Cypress.

For context, our extension aims to enhance the merchant discovery experience by monitoring any URLs on the web, meaning that for matches ("matches": ["http://*/*", "https://*/*"],) we use generic rules that even with the all_frames the extension will render within the window.top anyways.

You are correct that we could potentially overcome the issue to a certain extent by specifying a URL pattern which will skip Cypress localhost URL and force the extension to render within the iFrame, but this comes with the following price:

• Change the manifest to accommodate testing
• Changing the manifest from the testing phase to production (Remove all_frames and exclusions before publishing).

Additionally, as the extension is installed at the browser level, another thing to keep in mind when we use the webNavigation API is that we will have to discard the traffic from the main frame and listen to the iframe only during the acceptance tests.

chrome.webNavigation.onDOMContentLoaded.addListener(
    async (details: chrome.webNavigation.WebNavigationFramedCallbackDetails) => {
        // Only want to trigger an update when it's the parent frame (main page and not random iframes).
        const { tabId, frameId, url } = details
        if (frameId === 0) { <-- We will have to change this to support `all_frames` testing
            ...internal logic from the extension to process
        }
    }
)

Ideally, we would like Cypress to behave similarly to Puppeteer, where the test can run without the Cypress wrapper or side-by-side, like an extension within the browser. However, I understand the challenges of such a feature, but I am happy to brainstorm/contribute to such a feature if that helps the community towards testing chrome extensions.

[marktnoonan]

Appreciate the through explanation @eduardoacskimlinks. I'm going to track this as a feature request for now. I'm not sure we'll have anything to share in the short term, but we'll update this issue if there's any news.

[eduardoacskimlinks]

@marktnoonan, after our small chat, I feel this is a feature and a bug depending on how the Cypress team perceives chrome extension testing support.

I would like to propose for the short term to enhance the discussion by providing a bit more information on the limitations of Chrome extension testing. For example, it could even help developers to build extensions Cypress testing friendly.

Let me know if you see value in the proposal, and I can do some additional research and submit a pull request for the docs.

[cypress-app-bot]

This issue has not had any activity in 180 days. Cypress evolves quickly and the reported behavior should be tested on the latest version of Cypress to verify the behavior is still occurring. It will be closed in 14 days if no updates are provided.

[eduardoacskimlinks]

Yes, the issue is still present. It's up to the team if they want to keep or close the issue, as it will require changes in how Cypress works.

[jennifer-shehane]

Is this still an issue with the latest Cypress? Just wondering since there were some updates to extension support since we are running with --headless=new now in Chrome.

[eduardoacskimlinks]

Is this still an issue with the latest Cypress? I was just wondering since there were some updates to extension support since we are running with --headless=new now in Chrome.

This is still an issue; I haven't gotten back from the Cypress team, so we migrated to Jest + Puppeteer to move forward with our acceptance tests.