search

cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
46.93k stars 3.18k forks source link

E2E test spec fails at page forward - Receive "You seem to have gotten lost" message #24986

Open dcopening60 opened 1 year ago

dcopening60 commented 1 year ago

Current behavior

The Cypress test spec which failed consists of three H5P multiple choice questions, with one question per page. The first question writes a correct response. On the first page, when click on page forward, a blank page results.

Within the Cypress browser, the test spec exits and receive the message "You seem to have gotten lost...Try one of these links instead"

Recording: https://user-images.githubusercontent.com/61484854/205752619-2d86cce6-5fb5-4ac4-b5a5-3e79c18739cc.mp4

Desired behavior

Cypress-io should continue the test spec execution until the test exam is completed, as intended.

Steps - Observed changes in urls

  1. When accessing the first page of the exam, the url is: https://cypresstest2.sharmony.org/cypresstesttwo/multiple-choice-3-assessment-in-wordpress/ => The issue occurs when clicking on the page forward to page 2
  2. When page to page 2 via the page forward button, the url is: https://cypresstest2.sharmony.org/cypresstesttwo/multiple-choice-3-assessment-in-wordpress/#h5pbookid=1&section=top&chapter=h5p-interactive-book-chapter-d77ea472-c357-4846-9455-0332a6268113
  3. When page to page 3 via the page forward button, the url is: https://cypresstest2.sharmony.org/cypresstesttwo/multiple-choice-3-assessment-in-wordpress/#h5pbookid=1&section=top&chapter=h5p-interactive-book-chapter-e8c99065-51d4-490a-a11d-e69ffdf921c2

Test code to reproduce

https://github.com/dcopening60/H5P_Exam_Fail

Code snippet: 

...
  cy.visit("https://cypresstest2.sharmony.org/cypresstesttwo/multiple-choice-3-assessment-in-wordpress")
    cy.wait(4000)
    cy.frameLoaded('[data-content-id="' + 1 + '"]')       
    cy.task('log','iFrame loaded')

    //Question 1       
    cy.iframe('[data-content-id="' + 1 + '"]').find('[aria-labelledby="h5p-mcq0"]')
        .contains('State sovreignty, national union')
        .should('be.visible').click({ force: true, timeout: 40000 })
    cy.iframe('[data-content-id="' + 1 + '"]').contains('Check').click() 
    cy.wait(4000)
    //cy.pause()
    cy.iframe('[data-content-id="' + 1 + '"]')
        .find('[class="h5p-interactive-book-status"]')
        .find('[class="navigation-button icon-next"]')  //FAILS here
        .should('be.visible').first().click({force: true })    

    //Question 2       
...

Cypress Version

11.2.0

Node version

v14.15.0

Operating System

Windows 10 Enterprise OS build 19044.2251

Debug Logs

Debug Logs are too long to attach. The following json (file mochawesome_001.json) has error information:

{
  "stats": {
    "suites": 1,
    "tests": 0,
    "passes": 0,
    "pending": 0,
    "failures": 0,
    "start": "2022-12-05T23:09:19.413Z",
    "end": "2022-12-05T23:09:37.774Z",
    "duration": 18361,
    "testsRegistered": 1,
    "passPercent": 0,
    "pendingPercent": 0,
    "other": 0,
    "hasOther": false,
    "skipped": 1,
    "hasSkipped": true
  },
  "results": [
    {
      "uuid": "532b8586-3297-4b86-9216-6d42f5af1fed",
      "title": "",
      "fullFile": "cypress\\e2e\\004_WP_MultChoice3.cy.js",
      "file": "cypress\\e2e\\004_WP_MultChoice3.cy.js",
      "beforeHooks": [],
      "afterHooks": [],
      "tests": [],
      "suites": [
        {
          "uuid": "f5d06813-feda-485b-a199-0973ddfb9bd1",
          "title": "Student take H5P Interactive Book exam",
          "fullFile": "",
          "file": "",
          "beforeHooks": [],
          "afterHooks": [],
          "tests": [
            {
              "title": "Student take H5P Interactive Book exam ",
              "fullTitle": "Student take H5P Interactive Book exam Student take H5P Interactive Book exam ",
              "timedOut": null,
              "duration": 0,
              "state": "skipped",
              "speed": null,
              "pass": false,
              "fail": false,
              "pending": false,
              "context": null,
              "code": "Cypress.on('uncaught:exception', (err, runnable) => {\n  return false;\n});\ncy.visit(\"https://cypresstest2.sharmony.org/cypresstesttwo/multiple-choice-3-assessment-in-wordpress\");\ncy.wait(4000);\ncy.wait(4000);\ncy.frameLoaded('[data-content-id=\"' + 1 + '\"]');\ncy.task('log', 'iFrame loaded'); //Question 1\ncy.iframe('[data-content-id=\"' + 1 + '\"]').find('[aria-labelledby=\"h5p-mcq0\"]').contains('State sovreignty, national union').should('be.visible').click({\n  force: true,\n  timeout: 40000\n});\ncy.iframe('[data-content-id=\"' + 1 + '\"]').contains('Check').click();\ncy.wait(4000); //cy.pause()\ncy.iframe('[data-content-id=\"' + 1 + '\"]').find('[class=\"h5p-interactive-book-status\"]').find('[class=\"navigation-button icon-next\"]').should('be.visible').first().click({\n  force: true\n}); //Question 2       \ncy.iframe('[data-content-id=\"' + 1 + '\"]').find('[aria-labelledby=\"h5p-mcq1\"]').contains('Illinois').should('be.visible').click({\n  force: true,\n  timeout: 40000\n});\ncy.iframe('[data-content-id=\"' + 1 + '\"]').contains('Check').click();\ncy.wait(4000);\ncy.iframe('[data-content-id=\"' + 1 + '\"]').find('[class=\"h5p-interactive-book-status\"]').find('[class=\"navigation-button icon-next\"]').should('be.visible').first().click({\n  force: true\n}); //Question 3       \ncy.iframe('[data-content-id=\"' + 1 + '\"]').find('[aria-labelledby=\"h5p-mcq2\"]').contains('Illinois is the 21st state and Illinois joined the Union on December 1, 1818').should('be.visible').click({\n  force: true,\n  timeout: 40000\n});\ncy.iframe('[data-content-id=\"' + 1 + '\"]').contains('Check').click();\ncy.wait(4000);\ncy.iframe('[data-content-id=\"' + 1 + '\"]').find('[class=\"h5p-interactive-book-summary-text\"]').should('be.visible').first().click({\n  force: true\n});",
              "err": {},
              "uuid": "7287de73-f4a2-4796-a318-a811a2c164e4",  //=> error occurs here
              "parentUUID": "f5d06813-feda-485b-a199-0973ddfb9bd1",
              "isHook": false,
              "skipped": true
            }
          ],
          "suites": [],
          "passes": [],
          "failures": [],
          "pending": [],
          "skipped": [   
            "7287de73-f4a2-4796-a318-a811a2c164e4"  //=> error occurs here
          ],
          "duration": 0,
          "root": false,
          "rootEmpty": false,
          "_timeout": 2000
        }
      ],
      "passes": [],
      "failures": [],
      "pending": [],
      "skipped": [],
      "duration": 0,
      "root": true,
      "rootEmpty": true,
      "_timeout": 2000
    }
  ],
  "meta": {
    "mocha": {
      "version": "7.0.1"
    },
    "mochawesome": {
      "options": {
        "quiet": false,
        "reportFilename": "mochawesome",
        "saveHtml": false,
        "saveJson": true,
        "consoleReporter": "spec",
        "useInlineDiffs": false,
        "code": true
      },
      "version": "7.1.3"
    },
    "marge": {
      "options": {
        "overwrite": false,
        "html": false,
        "json": true,
        "reportDir": "cypress\\reports\\html\\.jsons"
      },
      "version": "6.2.0"
    }
  }
}

Other

Note regarding configuration: When the following is set to true, the data within the iframe does not display and the test spec fails:

export default defineConfig({
  "modifyObstructiveCode": true,

Receive a test failure filename mochawesome_003.json

Snag_f0f5c98

Need for solution to this issue: Page forward in an H5P exam needs to be testable within Cypress. Much of our system functionality depends on exam scores.

emilyrohrbough commented 1 year ago

@dcopening60 Thanks for proving a reproducible example. This was super helpful for diving into your issue....

So the reason you are seeing the "You seem to have gotten lost" page is because your application is busting out of the frame. The correct way to resolve this is to add "modifyObstructiveCode": true.

Unfortunately it seems there is a bug in the HTML parser we are leveraging that is used when modifyObstructiveCode=true. It seems it is parsing the text of Script tag

<script>H5PIntegration = {"baseUrl":"https:\/\/cypresstest2.sharmony.org\/cypresstesttwo","url":"\/cypresstesttwo\/wp-content\/uploads\/h5p","postUserStatistics":false,"ajax":
....

as

,\"library\":\"H5P.MultiChoice 1.16\",\"metadata\":{\"contentType\":\"Multiple Choice\",\"license\":\"U\",\"title\":\"Untitled Multiple Choice\"},\"subContentId\":\"176046c7-e9f7-4dc0-9740-e2f47ca4a736\"},\"useSeparator\":\"auto\"}]},\"library\":\"H5P.Column 1.15\",\"subContentId\":\"e8c99065-51d4-490a-a11d-e69ffdf921c2\",\"metadata\":{\"contentType\":\"Column\",\"license\":\"U\",\
...

which is why you are seeing the missing iframe on the page & the errors around , is invalid syntax in the console.

I spent quite a bit of time trying to see what can be done on our end, but we will need to log an issue to the parse5-html-rewriting-stream to correctly resolve this. I'll link the issue here once its logged.

Thank you for your patience. Sorry this is problematic!

emilyrohrbough commented 1 year ago

Issue logged with parse5-html-rewriting-stream: https://github.com/inikulin/parse5/issues/770.

dcopening60 commented 1 year ago

@emilyrohrbough thank you for researching this issue and logging it above. Is there an ETA when this issue might be resolved? Please let me know. thanks! Denise c.