sourcegraph / sourcegraph-public-snapshot

Code AI platform with Code Search & Cody
https://sourcegraph.com
Other
10.1k stars 1.28k forks source link

test:regression:core test fails: User settings are saved and applied #8470

Closed uwedeportivo closed 4 years ago

uwedeportivo commented 4 years ago

Steps to reproduce:

  1. IMAGE=sourcegraph/server:3.13.0-rc.1 ./dev/run-server-image.sh
  2. Configure external URL, add license, create sudo token
  3. source these env vars in your shell
export SOURCEGRAPH_BASE_URL='http://localhost:7080'
export SOURCEGRAPH_SUDO_TOKEN=xxxxxxx your token from above xxxxx
export SOURCEGRAPH_SUDO_USER=your instance user

you need a bunch more, they are in our 1password/shared/test:regression e2e environment variables:

https://team-sourcegraph.1password.com/vaults/dnrhbauihkhjs5ag6vszsme45a/allitems/gm5dfflq6sfclmotneuayfdj5q
  1. cd web
  2. yarn run test:regression:core

Expected behavior:

test suite passess

Actual behavior:

4 tests pass, 1 fails

Screen Shot 2020-02-17 at 6 07 19 PM

Core_functionality_regression_test_suite_2_2_1_User_settings_are_saved_and_applied

uwedeportivo commented 4 years ago

i get this one pretty consistently, so maybe it's legit:

~/work/src/github.com/sourcegraph/sourcegraph/web(3.13*) » yarn run test:regression:core                                                                                                                                                                                         uwe@maxx
yarn run v1.21.1
$ mocha ./src/regression/core.test.ts

  Core functionality regression test suite
    1) 2.2.1 User settings are saved and applied
📸  Saved screenshot of failure to ../puppeteer/Core_functionality_regression_test_suite_2_2_1_User_settings_are_saved_and_applied.png
    ✓ 2.2.2 User profile page (1381ms)
    ✓ 2.2.3. User emails page (1165ms)
    ✓ 2.2.4 Access tokens work and invalid access tokens return "401 Unauthorized" (2808ms)
    ✓ 2.5 Quicklinks: add a quicklink, test that it appears on the front page and works.
Test resource destroyed: Global setting "quicklinks"
    ✓ 2.4 Explore page

        @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
        @@@@ Manual verification steps required!!! @@@@
        @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

        Please verify the following screenshots match the corresponding descriptions:

        navbar-toggle-is-bart-simpson.png:  "Navbar toggle avatar is Bart Simpson"

  5 passing (25s)
  1 failing

  1) Core functionality regression test suite
       2.2.1 User settings are saved and applied:
     Error: Settings "{ /* These are new settings */}" did not match (old) saved settings "{\n      // add settings here (Ctrl+Space to see hints)\n}\n}"
      at Context.<anonymous> (src/regression/core.test.ts:143:19)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (internal/process/task_queues.js:93:5)
uwedeportivo commented 4 years ago

tested failing case by hand and it works, so test needs tweaks

uwedeportivo commented 4 years ago

Dear all,

This is your release captain speaking. 🚂🚂🚂

Branch cut for the 3.14 release is scheduled for tomorrow.

Is this issue / PR going to make it in time? Please change the milestone accordingly. When in doubt, reach out!

Thank you

uwedeportivo commented 4 years ago

fixed