rphlmr / supa-fly-stack

The Remix Stack for deploying to Fly with Supabase, authentication, testing, linting, formatting, etc.
MIT License
289 stars 26 forks source link

Possible issues with Cypress mocking? #34

Closed itsmetambui closed 1 year ago

itsmetambui commented 1 year ago

Hi again, not sure if I'm doing anything wrong but when I run the headless cypress test, the mocking doesn't seem to work and it calls my Supabase API. I can see the users and notes being created and cleaned up in my Supabase dashboard during the tests.

And since it called the actual api, test cases were failing and I had to add this check to wait for the api finishes and route changed to make it pass.

Screen Shot 2022-07-23 at 20 41 52

smoke tests -- should allow you to make a note (failed) smoke tests -- should allow you to register and login (failed)

Steps to reproduce:

  1. npx create-remix --template rphlmr/supa-fly-stack
  2. Correct the .env file
  3. npm run setup
  4. npm run test:e2e:run
rphlmr commented 1 year ago

Sorry for these troubles.

You are right, real api is call and I made a PR to remove mock server when running cypress. (You can still use It but It requires to mock every thing in your app 🥲. I use it for unit test) E2E tests must call real infrastructure to mimic real user experience.

For what you fix, yeah, I have some trouble with cypress and timeout (I made 30+ commits to test various workaround 😱). I can't reproduce with a fresh install :/

Can you tell me what you have in cypress.config.ts ?

rphlmr commented 1 year ago

Also, what is your operating system ? Maybe I should add what you propose 🧐

EDIT : This stack runs github action at every PR / merge master : https://github.com/rphlmr/supa-fly-stack/runs/7480850259?check_suite_focus=true Cypress makes me crasy :D

itsmetambui commented 1 year ago

Hi @rphlmr, thanks for checking this out.

Yeah, I feel weird to use mocks for e2e too. I guess the only issue left is the test cases fail. And yeah, same feeling for cypress :D. I'm using MacOS, I'm curious why https://github.com/rphlmr/supa-fly-stack/runs/7480850259?check_suite_focus=true didn't fail? And my cypress.config.ts doesn't have any change, I just pull the latest stack with RLS.

Here's all the checks I had to put in for the test to work.

Screen Shot 2022-07-23 at 21 13 54
rphlmr commented 1 year ago

@itsmetambui really curious, I'm on macOS too 😅

The reason why github action stop failing is that I add defaultCommandTimeout: process.env.CI ? 30_000 : 4_000 in https://github.com/rphlmr/supa-fly-stack/pull/32

Unless you have a CI var env somewhere (echo $CI should return nothing), I don't know what is happening.

Remix Grunge stack (dynamo db) doesn't mock anything in its E2E test (https://github.com/remix-run/grunge-stack/blob/main/cypress/e2e/smoke.cy.ts) or I miss something.

rphlmr commented 1 year ago

Maybe you can try to increase 4_000 to something else and tell me if it works ? If you validate that, I push a PR to increase It

itsmetambui commented 1 year ago

@rphlmr you're right, increase defaultCommandTimeout locally works, I increased it to 30_000

rphlmr commented 1 year ago

Nice ! I have merge to 30s. Thx