cypress-io / circleci-orb

Install, cache and run Cypress.io tests on CircleCI with minimal configuration.
https://circleci.com/orbs/registry/orb/cypress-io/cypress
MIT License
160 stars 101 forks source link

cypress/run has no executor param #429

Open Phonesis opened 1 year ago

Phonesis commented 1 year ago

It appears that with v3 of the orb you can no longer specify a specific executor for browsers when using the cypress/run command/job.

It appears to only be available for cypress/run-tests.

Is this intended? If so, the documentation is not clear or useful. For the install-browser param it says:

f you need additional browser support you can set this to false and use an executor with a docker image that includes the browsers of your choosing. See https://hub.docker.com/r/cypress/browsers/tags

How do you do this for cypress/run?

I would expect this to be valid:

      - cypress/run:
          name: cypress-uat-tests
          install-browsers: false
          executor: <my docker image> 
chrisbreiding commented 1 year ago

Did this used to work in v2? Looking at the previous implementation, seems like that aspect didn't change. In any case, I think this is a valid feature request.

I will forward this ticket to the appropriate team. They will evaluate the priority of this and consider their capacity to pick it up. Please note that this does not guarantee that this issue will be resolved. The ticket will indicate status changes during evaluation, so we ask that you please refrain from asking for updates. Thanks!

Phonesis commented 1 year ago

It did work with 2.2.0, we were using it. But it now fails to compile in CircleCI using v3.

As mentioned, it does work with cypress/run-tests though. But it should be possible for both job types.

jordanpowell88 commented 1 year ago

@Phonesis You can just specify your own executor for your job using something like https://github.com/cypress-io/circleci-orb/blob/master/.circleci/test-deploy.yml#L22.

Phonesis commented 1 year ago

@jordanpowell88 As mentioned, the problem resides with cypress/run

Your example is for cypress/run-tests

jordanpowell88 commented 1 year ago

Yes, cypress/run is meant to be the simplest out of the box solution (job) where the cypress/run-tests (command) is meant to provide people who want to hand roll their own solutions

Phonesis commented 1 year ago

Personally i still think cypress/run should have an executor option because many may want to use it but with a specific browser version in mind. Like we did.

Phonesis commented 1 year ago

Just a follow up to this, and why i think it is important to add to cypress/run.

install-browsers seems to fail a lot when you are running tests in parallel on multiple machines. We are constantly seeing it fail at the firefox install stage.

installing them all each time adds 1-2 minutes to each run as well. So simply using a defined executor is very important.

I have attempted to convert our pipeline to use cypress/run-tests. The problem I have now hit is that cypress/run-tests doesn't seem to include the following param:

  install-command

cypress/run does include this and we use it.

What is the equivalent for cypress/run-tests?

borkanee commented 1 year ago

Just a follow up to this, and why i think it is important to add to cypress/run.

install-browsers seems to fail a lot when you are running tests in parallel on multiple machines. We are constantly seeing it fail at the firefox install stage.

installing them all each time adds 1-2 minutes to each run as well. So simply using a defined executor is very important.

I have attempted to convert our pipeline to use cypress/run-tests. The problem I have now hit is that cypress/run-tests doesn't seem to include the following param:

  install-command

cypress/run does include this and we use it.

What is the equivalent for cypress/run-tests?

Did you ever solve this @Phonesis? Also using a custom executor in version 2 but failing in version 3.

Phonesis commented 1 year ago

@borkanee yes, solved it by using the start-command, which appears to do same thing / let you install stuff

      - cypress/run-tests:
          start-command: apt-get update && apt-get install curl -y #example
          cypress-command: <cypress script command>