Synthetixio / synpress

Synpress is e2e testing framework based on Cypress.io and playwright with support for metamask.
https://synpress.io
MIT License
565 stars 179 forks source link

cy.task('setupMetamask') issue #414

Open dlpigpen opened 2 years ago

dlpigpen commented 2 years ago

While running the setupMetamask i got this issue? Is there someone having the same problem?

setupMetamask, Object{3}
CypressError
cy.task('setupMetamask') failed with the following error:

> Cannot read properties of undefined (reading 'waitForTimeout')

Because this error occurred during a before each hook we are skipping the remaining tests in the current suite: wallet
smartdevcom commented 2 years ago

Yes, I got the same issue withsetupMetaMask

pcardosolei commented 2 years ago

what environemtn are you using it?

github actions?

smartdevcom commented 2 years ago

No, I used cypress with puppeteer

pcardosolei commented 2 years ago

I use the same as you.

waitForTimeout is because the puppeeteer cannot open in time.

What i did to overcome this was create a fake first test with wait(5000) to give time for pupeeteer to open up in time. Still trying to figure out what is going on in here.

P.S - i forked synpress to my repos and created also a npm package with the information need to work as a plugin.

you can take a look in here - https://github.com/pcardosolei/cypress-metamask-plugin

smartdevcom commented 2 years ago

I have checked this cypress-metamask-v2 repo as well and is it working well? Or did you update from this repo?

pcardosolei commented 2 years ago

Did not test that.

Totally forked from synpress. I want to have a stable plugin version and then pass the relevant information to this repo so people can use synpress on a more stable way.

The reason why i added as a plugin is to be flexible with my tests. I'm passing via CYPRESS_VARIABLE in the CI CD and cypress.env.json for the variables i want to have in my local tests.

smartdevcom commented 2 years ago

How could I integrate your repo into my project? Because I did yarn add -D cypress-metamask-v2 some days ago and it took very long time in link step so it looked very weird or heavy. Btw can I see your npm package link?

image

pcardosolei commented 2 years ago

https://www.npmjs.com/package/cypress-metamask-plugin

I will be updating the readMe as I have time.

smartdevcom commented 2 years ago

If you make any example without yarn add, it would very helpful

pcardosolei commented 2 years ago

on package json. "cy:run": "CYPRESS_REMOTE_DEBUGGING_PORT=9222 cypress run --browser chrome --headed"

I will update it.
pcardosolei commented 2 years ago

How could I integrate your repo into my project? Because I did yarn add -D cypress-metamask-v2 some days ago and it took very long time in link step so it looked very weird or heavy. Btw can I see your npm package link?

image

yes. i will be updating as I see the need.

nobita851 commented 2 years ago

While running the setupMetamask i got this issue? Is there someone having the same problem?

setupMetamask, Object{3}
CypressError
cy.task('setupMetamask') failed with the following error:

> Cannot read properties of undefined (reading 'waitForTimeout')

Because this error occurred during a before each hook we are skipping the remaining tests in the current suite: wallet

Is the issue resolved or did you come across any solution ?

pcardosolei commented 2 years ago

you tried on the local or github actions?

nobita851 commented 2 years ago

I tried on local. Facing this issue

image

My code:

image
marko911 commented 2 years ago

When I run synpress natively on my host, it works fine. But when I run it headed via xhost in a docker container, I get the same errors as above. Same thing if I run it headless in the docker container, I get these undefined / waitForTimeout errors.

Anyone have any luck tracking down what's actually going on ?

marko911 commented 2 years ago

I use the same as you.

waitForTimeout is because the puppeeteer cannot open in time.

What i did to overcome this was create a fake first test with wait(5000) to give time for pupeeteer to open up in time. Still trying to figure out what is going on in here.

P.S - i forked synpress to my repos and created also a npm package with the information need to work as a plugin.

you can take a look in here - https://github.com/pcardosolei/cypress-metamask-plugin

But the timeout error is happening in the before hook, the tests don't get a chance to even run.

pcardosolei commented 2 years ago

Push them to before each instead of before.

On Thu, 2 Jun 2022, 15:51 Marko Bilal, @.***> wrote:

I use the same as you.

waitForTimeout is because the puppeeteer cannot open in time.

What i did to overcome this was create a fake first test with wait(5000) to give time for pupeeteer to open up in time. Still trying to figure out what is going on in here.

P.S - i forked synpress to my repos and created also a npm package with the information need to work as a plugin.

you can take a look in here - https://github.com/pcardosolei/cypress-metamask-plugin

But the timeout error is happening in the before hook, the tests don't get a chance to even run.

— Reply to this email directly, view it on GitHub https://github.com/Synthetixio/synpress/issues/414#issuecomment-1144958136, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACNFYLU7EGJIYXLKU6HYFSLVNDDABANCNFSM5WGRF65Q . You are receiving this because you commented.Message ID: @.***>

marko911 commented 2 years ago

Push them to before each instead of before. On Thu, 2 Jun 2022, 15:51 Marko Bilal, @.> wrote: I use the same as you. waitForTimeout is because the puppeeteer cannot open in time. What i did to overcome this was create a fake first test with wait(5000) to give time for pupeeteer to open up in time. Still trying to figure out what is going on in here. P.S - i forked synpress to my repos and created also a npm package with the information need to work as a plugin. you can take a look in here - https://github.com/pcardosolei/cypress-metamask-plugin But the timeout error is happening in the before hook, the tests don't get a chance to even run. — Reply to this email directly, view it on GitHub <#414 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACNFYLU7EGJIYXLKU6HYFSLVNDDABANCNFSM5WGRF65Q . You are receiving this because you commented.Message ID: @.>

Ok I'll put it this way: Synpress run on native host : chrome opens , metamask setup runs right away and I see all the fields get populated for metamask setup

Synpress run from docker container: no metamask popup, just fails with the above errors as others have also posted.

pcardosolei commented 2 years ago

I created a single test to give time to pupeeteer to run everything. im still trying to put it from the start. I'm not using synpress . just it as a plugin.

dlpigpen commented 2 years ago

github actions?

Yes Github action does not work

ivelin commented 1 year ago

I tried on local. Facing this issue image

My code: image

Experiencing the same problem in Gitpod environment, which runs a remote docker image. The default puppeteer-spec.js tests from the Synpress repo pass. However metamask-spec.js from the same repo fail.

marko911 commented 1 year ago

I tried on local. Facing this issue image My code: image

Experiencing the same problem in Gitpod environment, which runs a remote docker image. The default puppeteer-spec.js tests from the Synpress repo pass. However metamask-spec.js from the same repo fail.

Is your localhost server running in the same container ? It might not be able to reach your web server

ivelin commented 1 year ago

Is your localhost server running in the same container ? It might not be able to reach your web server

Thank you for responding, @marko911

After tinkering quite a bit with environment settings and install dependencies, I was able to get to a point where synpress run works well, however synpress open shows the same type of errors.

Here is the latest repo: https://github.com/ivelin/sweat-token/tree/fix-pm-gas-v2/e2e-tests

Recording of a passing spec:

https://user-images.githubusercontent.com/2234901/184509727-b4580592-362f-4e97-8d6f-c3db912905d3.mp4

marko911 commented 1 year ago

@ivelin Why are you trying to do synpress open ?

ivelin commented 1 year ago

@ivelin Why are you trying to do synpress open ?

Just as a usability convenience. It saves some time from starting the whole cypress stack each time via synpress run.

qaultron commented 1 year ago

Hello! I have the same problem as described above run tests locally

describe('User can load page', () => {
        before(() => {
          cy.setupMetamask();
          cy.changeMetamaskNetwork('localhost')
          cy.visit('/')
        });
        it('is expected to display a sussess message', () => {
          cy.get('[data-cy=title]').should('contain.text', 'MetaMask Detected')
        });
        it('is expected to display the local wallet address', () => {
          cy.get('[data-cy=address').should('contain.text', 'Your address is: 0xf39fd6e51aad88f6f4ce6ab8827279cfffb92266')
        });
        it('is expected to display the local wallet  balance', () => {
          cy.get('[data-cy=balance').should('contain.text', 'Balance: 10000000000000000000000')
        });
      })
"@synthetixio/synpress": "^2.3.3-beta.0",
"cypress": "^10.8.0",

Screenshot_6

Super-Nim commented 1 year ago

Is your localhost server running in the same container ? It might not be able to reach your web server

Thank you for responding, @marko911

After tinkering quite a bit with environment settings and install dependencies, I was able to get to a point where synpress run works well, however synpress open shows the same type of errors.

Here is the latest repo: https://github.com/ivelin/sweat-token/tree/fix-pm-gas-v2/e2e-tests

Recording of a passing spec:

home.js.mp4

Hey, I am still experiencing the same error. What dependencies did you install to fix it?

marko911 commented 1 year ago

@Super-Nim I am not sure where you are at with your setup but I actually dug into the synpres code while they still used puppeteer and found a lot of hacky code with arbitrary timeouts like wait(3000) where the page would wait 3s for example to show the metamask modal. This wasn't enough time in my local setup so the metamask setup would always fail. Since then the synpress team has updated their library and use playwright instead of puppeteer. I'd try the new version if I were you and see if you still experience same problems.

Super-Nim commented 1 year ago

@Super-Nim I am not sure where you are at with your setup but I actually dug into the synpres code while they still used puppeteer and found a lot of hacky code with arbitrary timeouts like wait(3000) where the page would wait 3s for example to show the metamask modal. This wasn't enough time in my local setup so the metamask setup would always fail. Since then the synpress team has updated their library and use playwright instead of puppeteer. I'd try the new version if I were you and see if you still experience same problems.

I have tried both v1 and the latest versions of Synpress. The V1.2.0 loads the metamask setup tab - but then stops working after the first interaction.

I have been trying to get the Metamask setup tab to load for Synpress V2.4.1 but it just errors out like above, and doesn't even get setup. Here is my synpress.config.js file:

{
animationDistanceThreshold: 5,
arch: 'x64',
baseUrl: 'https://c3-my-test.myshopify.com',
blockHosts: null,
browsers: [
{
name: 'chrome',
family: 'chromium',
channel: 'stable',
displayName: 'Chrome',
version: '106.0.5249.103',
path: 'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe',
minSupportedVersion: 64,
majorVersion: '106',
},
{
name: 'edge',
family: 'chromium',
channel: 'stable',
displayName: 'Edge',
version: '106.0.1370.37',
path: 'C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe',
minSupportedVersion: 79,
majorVersion: '106',
},
{
name: 'electron',
channel: 'stable',
family: 'chromium',
displayName: 'Electron',
version: '102.0.5005.148',
path: '',
majorVersion: 102,
},
],
chromeWebSecurity: true,
clientCertificates: [],
defaultCommandTimeout: 300000,
downloadsFolder: 'cypress/downloads',
env: {
coverage: false,
REMOTE_DEBUGGING_PORT: 9222,
},
excludeSpecPattern: '*.hot-update.js',
execTimeout: 60000,
experimentalFetchPolyfill: false,
experimentalInteractiveRunEvents: false,
experimentalModifyObstructiveThirdPartyCode: false,
experimentalSessionAndOrigin: false,
experimentalSingleTabRunMode: false,
experimentalSourceRewriting: false,
experimentalStudio: false,
experimentalWebKitSupport: false,
fileServerFolder: '',
fixturesFolder: 'cypress/fixtures',
hosts: null,
includeShadowDom: false,
isInteractive: true,
keystrokeDelay: 0,
modifyObstructiveCode: true,
nodeVersion: null,
numTestsKeptInMemory: 50,
pageLoadTimeout: 300000,
platform: 'win32',
port: null,
projectId: null,
redirectionLimit: 20,
reporter: 'spec',
reporterOptions: null,
requestTimeout: 300000,
resolvedNodePath: null,
resolvedNodeVersion: null,
responseTimeout: 30000,
retries: {
runMode: 0,
openMode: 0,
},
screenshotOnRunFailure: true,
screenshotsFolder: 'cypress/screenshots',
scrollBehavior: 'top',
slowTestThreshold: 10000,
specPattern: 'cypress/e2e/**/*.cy.{js,jsx,ts,tsx}',
supportFile: 'cypress/support/index.js',
supportFolder: false,
taskTimeout: 60000,
testIsolation: 'legacy',
trashAssetsBeforeRuns: true,
userAgent: 'synpress',
video: false,
videoCompression: 32,
videosFolder: 'videos',
videoUploadOnPasses: true,
viewportHeight: 850,
viewportWidth: 1366,
waitForAnimations: true,
watchForFileChanges: true,
}
doliG commented 1 year ago

Hey there,

When I use synpress run it's working fine, however when I use synpress open I get the same error as @qaultron and @nobita851 (see screenshot below).

I think I'm using the latest version (2.4.1-beta.1), so it's probably not due to a pupetter hack as pointed out by @marko911 .

Using the open mode is, as @ivelin said, a must have for convenience and a requirement for me.

Any clues on this ? Thanks :)

image
Christopher-Stevers commented 1 year ago

image Having the same issue as @doliG, its running fine when I do synpress run though.

hadothanh commented 1 year ago

I got the same issue when tried using synpress as cypress plugin with typescript, but work normally in Javascript

YakovL commented 1 year ago

@hadothanh could you clarify switching which files from JS to TS or vise versa breaks/heals things?

YakovL commented 1 year ago

Like for @doliG and for @ivelin, the same issue happens for me with @synthetixio/synpress@3.7.1 and Cypress@12.14: cypress run works fine while cypress open fails. This is a problem, since the watch mode in cypress open creates nice feedback loop when writing tests, while cypress run makes it much slower.

PS by the way, this problem is described at #417

PPS Another issue with cypress run mode is: the code that needs debugging is not reachable. For instance, when I get:

     ReferenceError: The following error originated from your test code, not from Cypress.

  > process is not defined

When Cypress detects uncaught errors originating from your test code it will automatically fail the current test.
      at $Cypress.eval (https://market-testnet.iq.space/__cypress/tests?p=cypress\support\e2e.ts:17196:9)

the https://market-testnet.iq.space/__cypress/tests?p=cypress\support\e2e.ts:17196:9 url isn't in fact a real url. When in the cypress open mode, I can use Chrome dev tools to inspect the code; in the run mode, in contrast, I can't do that.

Joshua-onwuzu commented 1 year ago

is anybody able to get synpress to work in git actions

matiasdaloia commented 12 months ago

Hi any news about this?

doliG commented 12 months ago

I gave up ( ´_ゝ`)

YakovL commented 11 months ago

Hey guys, please take a look at the workaround I've mentioned here: if you are getting process is not defined errors, open the node_modules/@synthetixio/synpress/support/index.js and try commenting out the following lines: 8, 10, 17-19:

// dont fail tests on uncaught exceptions of websites
Cypress.on('uncaught:exception', () => {
  //if (!process.env.FAIL_ON_ERROR) {
    return false;
  //}
});

Cypress.on('window:before:load', win => {
  cy.stub(win.console, 'error').callsFake(message => {
    cy.now('task', 'error', message);
    // fail test on browser console error
    //if (process.env.FAIL_ON_ERROR) {
    //  throw new Error(message);
    //}
  });

  cy.stub(win.console, 'warn').callsFake(message => {
    cy.now('task', 'warn', message);
  });
});

(alternatively, if you want the FAIL_ON_ERROR behavior, don't comment out the line 18 and comment out the line 9)

Let me know if that removes process is not defined errors (and may be some others?) for you and/or shows more reasonable errors instead!

PS For me, after commenting those bits out, for cypress open I got

cy.task('setupMetamask') failed with the following error: request to http://127.0.0.1:9222/json/version failed, reason: connect ECONNREFUSED 127.0.0.1:9222

which is an error that I know already, so I've made sure that my .env.synpress is passed to cypress open (env-cmd -f .env.synpress cypress open, with CYPRESS_REMOTE_DEBUGGING_PORT=9222 in .env.synpress). After that, I can see that setupMetamask now takes effect, although I'm having troubles with "Incorrect password" (for some reason, setupMetamask is run more than once, any the password field is filled incorrectly).

The good news is, my PR fixing this bit was merged, so this will be probably fixed in 3.7.2.

izuchukwu-eric commented 4 months ago

Hi @YakovL I used to get the below error when using cypress: 12.7.3

cy.task('setupMetamask') failed with the following error: request to http://127.0.0.1:9222/json/version failed, reason: connect ECONNREFUSED 127.0.0.1:9222

But then i upgraded to cypress: ^13.7.0 and i keep getting this error when i try to run cypress:

Webpack Compilation Error Module parse failed: 'import' and 'export' may appear only with 'sourceType: module' (1:0) You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://> webpack.js.org/concepts#loaders import './commands'; import { configure } from '@testing-library/cypress'

Would appriciate any tip on how to fix this

YakovL commented 4 months ago

Hi @izuchukwu-eric , what file does the error come from? I haven't encountered this, but it looks like you have to configure your Webpack (to recognize ES modules correctly)