cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
47.01k stars 3.18k forks source link

CYPRESS_ env variable string value containing commas parsed as multiple values #8818

Closed bryguy closed 3 years ago

bryguy commented 4 years ago

Current behavior

After upgrading Cypress from 4.10.0 to 5.3.0, one of my tests started failing. This appears to be due to a change in the way CYPRESS_ environment variables are getting parsed between the two versions.

I'm passing a string of JSON data to my test via a CYPRESS_ prefixed environment variable like so:

CYPRESS_stringified_json='[{"type": "foo", "value": "bar"}, {"type": "fizz", "value": "buzz"}]' node_modules/.bin/cypress run --spec cypress/integration/spec.js

When attempting to call JSON.parse(Cypress.env('stringified_json')) on that value in my test, the test started failing with the following error:

 > Unexpected token , in JSON at position ###

After digging in a little bit, I found that instead of the expected JSON string, the value for that variable now contains an array of the passed string value split on every comma. Here's the output for 5.3.0 using the simplified test code attached below:

====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:    5.3.0                                                                              │
  │ Browser:    Electron 83 (headless)                                                             │
  │ Specs:      1 found (spec.js)                                                                  │
  │ Searched:   cypress/integration/spec.js                                                        │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  spec.js                                                                         (1 of 1)

  process environment variables
    1) correctly parses stringified JSON data

  0 passing (309ms)
  1 failing

  1) process environment variables
       correctly parses stringified JSON data:

      AssertionError: expected [ Array(4) ] to equal '[{"type": "foo", "value": "bar"}, {"type": "fizz", "value": "buzz"}]'
      + expected - actual

      -[ '{"type": "foo"',
      -  ' "value": "bar"}',
      -  ' {"type": "fizz"',
      -  ' "value": "buzz"}' ]
      +'[{"type": "foo", "value": "bar"}, {"type": "fizz", "value": "buzz"}]'

      at Context.eval (http://localhost:34691/__cypress/tests?p=cypress/integration/spec.js:101:48)

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        1                                                                                │
  │ Passing:      0                                                                                │
  │ Failing:      1                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  1                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     0 seconds                                                                        │
  │ Spec Ran:     spec.js                                                                          │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

  (Screenshots)

  -  /cypress/screenshots/spec.js/process environment variables     (1280x720)
      -- correctly parses stringified JSON data (failed).png                                        

====================================================================================================

  (Run Finished)

       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✖  spec.js                                  300ms        1        -        1        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✖  1 of 1 failed (100%)                     300ms        1        -        1        -        -  

Desired behavior

Simply pass through the value for the environment variable without splitting on commas the same way it used to be passed in 4.10.0. Here's the output of the test code attached below for 4.10.0:

====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:    4.10.0                                                                             │
  │ Browser:    Electron 80 (headless)                                                             │
  │ Specs:      1 found (spec.js)                                                                  │
  │ Searched:   cypress/integration/spec.js                                                        │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

────────────────────────────────────────────────────────────────────────────────────────────────────

  Running:  spec.js                                                                         (1 of 1)

  0 passing (1ms)

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        0                                                                                │
  │ Passing:      0                                                                                │
  │ Failing:      0                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     0 seconds                                                                        │
  │ Spec Ran:     spec.js                                                                          │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

====================================================================================================

  (Run Finished)

       Spec                                              Tests  Passing  Failing  Pending  Skipped
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  spec.js                                    1ms        -        -        -        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                          1ms        -        -        -        -        -

Test code to reproduce

// CYPRESS_stringified_json='[{"type": "foo", "value": "bar"}, {"type": "fizz", "value": "buzz"}]' node_modules/.bin/cypress run --spec cypress/integration/spec.js

describe('process environment variables', () => {
    it('correctly parses stringified JSON data', () => {
        expect(Cypress.env('stringified_json')).to.equal(
            '[{"type": "foo", "value": "bar"}, {"type": "fizz", "value": "buzz"}]'
        );
    });
});

Versions

Cypress: Cypress 5.3.0 Operating System: macOS 10.15.7 Browser: Electron 83 (headless)

bryguy commented 4 years ago

As a temporary workaround, I updated the actual test that started breaking with the following hack to unmangle the value back into its original stringified JSON format like so:

const test_data = JSON.parse('[' + Cypress.env('stringified_json').join(',') + ']');
jennifer-shehane commented 4 years ago

This issue was introduced in 5.0.0.

Reproducible example

CYPRESS_stringified_json='[{"type": "foo", "value": "bar"}, {"type": "fizz", "value": "buzz"}]' cypress open

spec.js

it('correctly parses stringified JSON data', () => {
  expect(Cypress.env('stringified_json')).to.equal(
    '[{"type": "foo", "value": "bar"}, {"type": "fizz", "value": "buzz"}]'
  )
})

4.12.1

Screen Shot 2020-10-13 at 11 57 34 AM

5.0.0

Screen Shot 2020-10-13 at 12 40 40 PM

Why

I'm sure this PR likely introduced this: https://github.com/cypress-io/cypress/pull/8151 We would be open to a PR to fix this.

sainthkh commented 3 years ago

The requested change was to return the original JSON string. But the goal of #8151 was to parse and return a useful value from Cypress side. So, I decided to parse the JSON string to object.

cypress-bot[bot] commented 3 years ago

The code for this is done in cypress-io/cypress#9584, but has yet to be released. We'll update this issue and reference the changelog when it's released.

cypress-bot[bot] commented 3 years ago

Released in 6.3.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to Cypress v6.3.0, please open a new issue.