search

cypress-io / circleci-orb

Install, cache and run Cypress.io tests on CircleCI with minimal configuration.
https://circleci.com/orbs/registry/orb/cypress-io/cypress
MIT License
159 stars 101 forks source link

Install ChromeDriver 404 error with install-browsers flag #460

Closed johnkim-det closed 4 months ago

johnkim-det commented 4 months ago

The orb does not work as expected

orbs: cypress: cypress-io/cypress@3.3.0

workflows: main: jobs:

Chrome version major is 122
Installed version of Google Chrome is 122.0.6261.94 
404
Matching Chrome Driver Version 404'd, falling back to first matching major version.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   937  100   937    0     0  13573      0 --:--:-- --:--:-- --:--:-- 13779
New ChromeDriver version to be installed: 122.0.6261.94
122.0.6261.94 will be installed
curl: (22) The requested URL returned error: 404

Exited with code exit status 22

Tried multiple versions (3.3.0, 3.2.1, 3.2.0, 3.1.4, 3.1.3), all with the same error.

jennifer-shehane commented 4 months ago

This was caused by an issue with browser-tools orb which is a dependency of the cypress orb. It was fixed here: https://github.com/CircleCI-Public/browser-tools-orb/pull/105

We set browser-tools: 1 as the version, so my expectation is that this would update to latest underneath. I'm not completely sure how the versioning works for the orbs with interdependent orbs.

MikeMcC399 commented 4 months ago

@jennifer-shehane

This is impacting https://github.com/cypress-io/cypress-example-kitchensink again. (See https://github.com/cypress-io/cypress-example-kitchensink/pull/797).

MikeMcC399 commented 4 months ago

@jennifer-shehane

This was caused by an issue with browser-tools orb which is a dependency of the cypress orb. It was fixed here: CircleCI-Public/browser-tools-orb#105

We set browser-tools: 1 as the version, so my expectation is that this would update to latest underneath. I'm not completely sure how the versioning works for the orbs with interdependent orbs.

CircleCI documentation states that published orbs are immutable which ties in with the observation that the old version circleci/browser-tools@1.4.6 is being used by cypress-io/cypress@3.3.0. This was the current version of circleci/browser-tools at the time that cypress-io/cypress@3.3.0 was published.

Suggestion

Bump the version of circleci/browser-tools to circleci/browser-tools@1.4.8 in

https://github.com/cypress-io/circleci-orb/blob/a5e724b478e33e5eb3c679dfabab9bd488eb4927/src/%40orb.yml#L13

replacing the semver @1 with an explicit version @1.4.8

and republish the orb.

It seems that the version of circleci/browser-tools is only re-evaluated when the cypress-io/cypress orb is republished and not at run-time when the orb is used. Listing @1 is confusing since in that case neither GitHub nor CircleCI apparently documents exactly which version of circleci/browser-tools has been pulled in.

In comparison to GitHub workflows this is counter-intuitive. Nested workflows of JavaScript actions calling other JavaScript actions evaluate versions to be used at run-time.

cc: @DaleGardner

DaleGardner commented 4 months ago

That makes sense, great explanation Mike!

MikeMcC399 commented 4 months ago

@jennifer-shehane

https://discuss.circleci.com/t/orb-to-orb-dependency-resolution/29012 from 2019 says

"In short—orbs are designed to be immutable, so when you import one orb in another orb, you import that orb at a specific version, which will never change unless you manually change it."

however that does not seem to be particularly clear reading the documentation

semantic versioning

DaleGardner commented 4 months ago

PR opened to fix this issue. @jennifer-shehane If you can review, it would be much appreciated! Then hopefully could get a new version published and resolve the error! Thank you @MikeMcC399 for pointing this out and all the research!!

MikeMcC399 commented 4 months ago

@DaleGardner

jennifer-shehane commented 4 months ago

@MikeMcC399 Annoying. Thanks for diving into this and assessing the true state of how it works. I was confused by the semver allowance also, yet it doesn't abide by it.

jennifer-shehane commented 4 months ago

Please update to cypress' 3.3.1 orb to get this fix: https://github.com/cypress-io/circleci-orb/releases/tag/v3.3.1