search

inikulin / parse5

HTML parsing/serialization toolset for Node.js. WHATWG HTML Living Standard (aka HTML5)-compliant.
MIT License
3.68k stars 236 forks source link

Incorrectly parsing text from script #770

Open emilyrohrbough opened 1 year ago

emilyrohrbough commented 1 year ago

We are using the parse5-html-rewriting-stream, to parse html & js body content from intercepted browser requests.

function rewriteHtmlJs (url, html, deferSourceMapRewrite) {
  let out = ''
  const rewriter = new RewritingStream()

  rewriter.on('startTag', (startTag, raw) => { })

  rewriter.on('endTag', (_endTag, raw) => { })

  rewriter.on('text', (_textToken, raw) => { })

  rewriter.on('data', (chunk) => {
    out += chunk
  })

  rewriter.end(html)

  return new Promise((resolve) => {
    rewriter.on('end', () => {
      resolve(out)
    })
  })
}

This work well, but recently stumbled across an issue where the following script content were being incorrectly/partially emitted when the internal contents were JS setting a constant to a JSON.

<script>H5PIntegration = {"baseUrl":"example.com,"url":"example.com,"postUserStatistics":false,"ajax":{"setFinished":"example.com\/wp-admin\/admin-ajax.php?token=8afbbf343c&action=h5p_setFinished","contentUserData":"example.com\/wp-admin\/admin-ajax.php?token=252f1904de&action=h5p_contents_user_data&content_id=:contentId&data_type=:dataType&sub_content_id=:subContentId"},"saveFreq":false,"siteUrl":"exampe.com","l10n":{"H5P":{"fullscreen":"Fullscreen","disableFullscreen":"Disable fullscreen","download":"Download","copyrights":"Rights of use","embed":"Embed","size":"Size","showAdvanced":"Show advanced","hideAdvanced":"Hide advanced","advancedHelp":"Include this script on your website if you want dynamic sizing of the embedded content:","copyrightInformation":"Rights of use","close":"Close","title":"Title","author":"Author","year":"Year","source":"Source","license":"License","thumbnail":"Thumbnail","noCopyrights":"No copyright information available for this content.","reuse":"Reuse","reuseContent":"Reuse Content","reuseDescription":"Reuse this content.","downloadDescription":"Download this content as a H5P file.","copyrightsDescription":"View copyright information for this content.","embedDescription":"View the embed code for this content.","h5pDescription":"Visit H5P.org to check out more cool content.","contentChanged":"This content has changed since you last used it.","startingOver":"You'll be starting over.","by":"by","showMore":"Show more","showLess":"Show less","subLevel":"Sublevel","confirmDialogHeader":"Confirm action","confirmDialogBody":"Please confirm that you wish to proceed. This action is not reversible.","cancelLabel":"Cancel","confirmLabel":"Confirm","licenseU":"Undisclosed","licenseCCBY":"Attribution","licenseCCBYSA":"Attribution-ShareAlike","licenseCCBYND":"Attribution-NoDerivs","licenseCCBYNC":"Attribution-NonCommercial","licenseCCBYNCSA":"Attribution-NonCommercial-ShareAlike","licenseCCBYNCND":"Attribution-NonCommercial-NoDerivs","licenseCC40":"4.0 International","licenseCC30":"3.0 Unported","licenseCC25":"2.5 Generic","licenseCC20":"2.0 Generic","licenseCC10":"1.0 Generic","licenseGPL":"General Public License","licenseV3":"Version 3","licenseV2":"Version 2","licenseV1":"Version 1","licensePD":"Public Domain","licenseCC010":"CC0 1.0 Universal (CC0 1.0) Public Domain Dedication","licensePDM":"Public Domain Mark","licenseC":"Copyright","contentType":"Content Type","licenseExtras":"License Extras","changes":"Changelog","contentCopied":"Content is copied to the clipboard","connectionLost":"Connection lost. Results will be stored and sent when you regain connection.","connectionReestablished":"Connection reestablished.","resubmitScores":"Attempting to submit stored results.","offlineDialogHeader":"Your connection to the server was lost","offlineDialogBody":"We were unable to send information about your completion of this task. Please check your internet connection.","offlineDialogRetryMessage":"Retrying in :num....","offlineDialogRetryButtonLabel":"Retry now","offlineSuccessfulSubmit":"Successfully submitted results."}},"hubIsEnabled":true,"reportingIsEnabled":false,"libraryConfig":null,"crossorigin":null,"crossoriginCacheBuster":null,"pluginCacheBuster":"?v=1.15.6","libraryUrl":"example.com\/wp-content\/plugins\/h5p\/h5p-php-library\/js","core":{"styles":["\/cypresstesttwo\/wp-content\/plugins\/h5p\/h5p-php-library\/styles\/h5p.css?ver=1.15.6","\/cypresstesttwo\/wp-content\/plugins\/h5p\/h5p-php-library\/styles\/h5p-confirmation-dialog.css?ver=1.15.6","\/cypresstesttwo\/wp-content\/plugins\/h5p\/h5p-php-library\/styles\/h5p-core-button.css?ver=1.15.6"],"scripts":["\/cypresstesttwo\/wp-content\/plugins\/h5p\/h5p-php-library\/js\/jquery.js?ver=1.15.6","\/cypresstesttwo\/wp-content\/plugins\/h5p\/h5p-php-library\/js\/h5p.js?ver=1.15.6","\/cypresstesttwo\/wp-content\/plugins\/h5p\/h5p-php-library\/js\/h5p-event-dispatcher.js?ver=1.15.6","\/cypresstesttwo\/wp-content\/plugins\/h5p\/h5p-php-library\/js\/h5p-x-api-event.js?ver=1.15.6","\/cypresstesttwo\/wp-content\/plugins\/h5p\/h5p-php-library\/js\/h5p-x-api.js?ver=1.15.6","\/cypresstesttwo\/wp-content\/plugins\/h5p\/h5p-php-library\/js\/h5p-content-type.js?ver=1.15.6","\/cypresstesttwo\/wp-content\/plugins\/h5p\/h5p-php-library\/js\/h5p-confirmation-dialog.js?ver=1.15.6","\/cypresstesttwo\/wp-content\/plugins\/h5p\/h5p-php-library\/js\/h5p-action-bar.js?ver=1.15.6","\/cypresstesttwo\/wp-content\/plugins\/h5p\/h5p-php-library\/js\/request-queue.js?ver=1.15.6"]},"loadedJs":[],"loadedCss":[],"contents":{"cid-1":{"library":"H5P.InteractiveBook 1.6","jsonContent":"{\"showCoverPage\":false,\"bookCover\":{\"coverDescription\":\"<p style=\\\"text-align: center;\\\"><\\\/p>\"},\"chapters\":[{\"params\":{\"content\":[{\"content\":{\"params\":{\"media\":{\"disableImageZooming\":false},\"answers\":[{\"correct\":false,\"tipsAndFeedback\":{\"tip\":\"\",\"chosenFeedback\":\"\",\"notChosenFeedback\":\"\"},\"text\":\"<div>Forward<\\\/div>\\n\"},{\"correct\":true,\"tipsAndFeedback\":{\"tip\":\"\",\"chosenFeedback\":\"\",\"notChosenFeedback\":\"\"},\"text\":\"<div>State sovreignty, national union<\\\/div>\\n\"},{\"correct\":false,\"tipsAndFeedback\":{\"tip\":\"\",\"chosenFeedback\":\"\",\"notChosenFeedback\":\"\"},\"text\":\"<div>Crossroads of America<\\\/div>\\n\"},{\"correct\":false,\"tipsAndFeedback\":{\"tip\":\"\",\"chosenFeedback\":\"\",\"notChosenFeedback\":\"\"},\"text\":\"<div>Our liberties we prize, and our rights we maintain<\\\/div>\\n\"},{\"correct\":false,\"tipsAndFeedback\":{\"tip\":\"\",\"chosenFeedback\":\"\",\"notChosenFeedback\":\"\"},\"text\":\"<div>With God, all things are possible<\\\/div>\\n\"}],\"overallFeedback\":[{\"from\":0,\"to\":74,\"feedback\":\"Keep going...\"},{\"from\":75,\"to\":100,\"feedback\":\"You&#039;ve got it!\"}],\"behaviour\":{\"enableRetry\":true,\"enableSolutionsButton\":true,\"enableCheckButton\":true,\"type\":\"auto\",\"singlePoint\":false,\"randomAnswers\":true,\"showSolutionsRequiresInput\":true,\"confirmCheckDialog\":false,\"confirmRetryDialog\":false,\"autoCheck\":false,\"passPercentage\":100,\"showScorePoints\":true},\"UI\":{\"checkAnswerButton\":\"Check\",\"submitAnswerButton\":\"Submit\",\"showSolutionButton\":\"Show solution\",\"tryAgainButton\":\"Retry\",\"tipsLabel\":\"Show tip\",\"scoreBarLabel\":\"You got :num out of :total points\",\"tipAvailable\":\"Tip available\",\"feedbackAvailable\":\"Feedback available\",\"readFeedback\":\"Read feedback\",\"wrongAnswer\":\"Wrong answer\",\"correctAnswer\":\"Correct answer\",\"shouldCheck\":\"Should have been checked\",\"shouldNotCheck\":\"Should not have been checked\",\"noInput\":\"Please answer before viewing the solution\",\"a11yCheck\":\"Check the answers. The responses will be marked as correct, incorrect, or unanswered.\",\"a11yShowSolution\":\"Show the solution. The task will be marked with its correct solution.\",\"a11yRetry\":\"Retry the task. Reset all responses and start the task over again.\"},\"confirmCheck\":{\"header\":\"Finish ?\",\"body\":\"Are you sure you wish to finish ?\",\"cancelLabel\":\"Cancel\",\"confirmLabel\":\"Finish\"},\"confirmRetry\":{\"header\":\"Retry ?\",\"body\":\"Are you sure you wish to retry ?\",\"cancelLabel\":\"Cancel\",\"confirmLabel\":\"Confirm\"},\"question\":\"<p>What is the Illinois state motto?<\\\/p>\\n\"},\"library\":\"H5P.MultiChoice 1.16\",\"metadata\":{\"contentType\":\"Multiple Choice\",\"license\":\"U\",\"title\":\"Illinois State Motto\"},\"subContentId\":\"e2bc7ad1-7a40-4925-a60b-aae393ef551e\"},\"useSeparator\":\"auto\"}]},\"library\":\"H5P.Column 1.15\",\"subContentId\":\"8fd7416d-831f-432f-95bf-58cee8466cb2\",\"metadata\":{\"contentType\":\"Column\",\"license\":\"U\",\"title\":\"Illinois State Motto\"}},{\"params\":{\"content\":[{\"content\":{\"params\":{\"media\":{\"disableImageZooming\":false},\"answers\":[{\"correct\":false,\"tipsAndFeedback\":{\"tip\":\"\",\"chosenFeedback\":\"\",\"notChosenFeedback\":\"\"},\"text\":\"<div>Rocky Mountain High<\\\/div>\\n\"},{\"correct\":true,\"tipsAndFeedback\":{\"tip\":\"\",\"chosenFeedback\":\"\",\"notChosenFeedback\":\"\"},\"text\":\"<div>Illinois<\\\/div>\\n\"},{\"correct\":false,\"tipsAndFeedback\":{\"tip\":\"\",\"chosenFeedback\":\"\",\"notChosenFeedback\":\"\"},\"text\":\"<div>On, Wisconsin!<\\\/div>\\n\"},{\"correct\":false,\"tipsAndFeedback\":{\"tip\":\"\",\"chosenFeedback\":\"\",\"notChosenFeedback\":\"\"},\"text\":\"<div>On the Banks of the Wabash<\\\/div>\\n\"}],\"overallFeedback\":[{\"from\":0,\"to\":74,\"feedback\":\"Keep going...\"},{\"from\":75,\"to\":100,\"feedback\":\"You&#039;ve got it!\"}],\"behaviour\":{\"enableRetry\":true,\"enableSolutionsButton\":true,\"enableCheckButton\":true,\"type\":\"auto\",\"singlePoint\":false,\"randomAnswers\":true,\"showSolutionsRequiresInput\":true,\"confirmCheckDialog\":false,\"confirmRetryDialog\":false,\"autoCheck\":false,\"passPercentage\":100,\"showScorePoints\":true},\"UI\":{\"checkAnswerButton\":\"Check\",\"submitAnswerButton\":\"Submit\",\"showSolutionButton\":\"Show solution\",\"tryAgainButton\":\"Retry\",\"tipsLabel\":\"Show tip\",\"scoreBarLabel\":\"You got :num out of :total points\",\"tipAvailable\":\"Tip available\",\"feedbackAvailable\":\"Feedback available\",\"readFeedback\":\"Read feedback\",\"wrongAnswer\":\"Wrong answer\",\"correctAnswer\":\"Correct answer\",\"shouldCheck\":\"Should have been checked\",\"shouldNotCheck\":\"Should not have been checked\",\"noInput\":\"Please answer before viewing the solution\",\"a11yCheck\":\"Check the answers. The responses will be marked as correct, incorrect, or unanswered.\",\"a11yShowSolution\":\"Show the solution. The task will be marked with its correct solution.\",\"a11yRetry\":\"Retry the task. Reset all responses and start the task over again.\"},\"confirmCheck\":{\"header\":\"Finish ?\",\"body\":\"Are you sure you wish to finish ?\",\"cancelLabel\":\"Cancel\",\"confirmLabel\":\"Finish\"},\"confirmRetry\":{\"header\":\"Retry ?\",\"body\":\"Are you sure you wish to retry ?\",\"cancelLabel\":\"Cancel\",\"confirmLabel\":\"Confirm\"},\"question\":\"<p>What is the Illinois state song?<\\\/p>\\n\"},\"library\":\"H5P.MultiChoice 1.16\",\"metadata\":{\"contentType\":\"Multiple Choice\",\"license\":\"U\",\"title\":\"Illinois state song\"},\"subContentId\":\"1d985b18-3ee1-40ee-8732-ed9f468d4304\"},\"useSeparator\":\"auto\"}]},\"library\":\"H5P.Column 1.15\",\"subContentId\":\"d77ea472-c357-4846-9455-0332a6268113\",\"metadata\":{\"contentType\":\"Column\",\"license\":\"U\",\"title\":\"Illinois state song\"}},{\"params\":{\"content\":[{\"content\":{\"params\":{\"media\":{\"disableImageZooming\":false},\"answers\":[{\"correct\":true,\"tipsAndFeedback\":{\"tip\":\"\",\"chosenFeedback\":\"\",\"notChosenFeedback\":\"\"},\"text\":\"<div>Illinois is the 21st state and Illinois joined the Union on December 1, 1818<\\\/div>\\n\"},{\"correct\":false,\"tipsAndFeedback\":{\"tip\":\"\",\"chosenFeedback\":\"\",\"notChosenFeedback\":\"\"},\"text\":\"<div>Illinois is the 20th state and Illinois joined the Union on December 10,1817<\\\/div>\\n\"},{\"correct\":false,\"tipsAndFeedback\":{\"tip\":\"\",\"chosenFeedback\":\"\",\"notChosenFeedback\":\"\"},\"text\":\"<div>Illinois is the 19th state and Illinois joined the Union on December 11,1816<\\\/div>\\n\"},{\"correct\":false,\"tipsAndFeedback\":{\"tip\":\"\",\"chosenFeedback\":\"\",\"notChosenFeedback\":\"\"},\"text\":\"<div>Illinois is the 22th state and Illinois joined the Union on December 14,1819<\\\/div>\\n\"}],\"overallFeedback\":[{\"from\":0,\"to\":74,\"feedback\":\"Keep going...\"},{\"from\":75,\"to\":100,\"feedback\":\"You&#039;ve got it!\\u2063\"}],\"behaviour\":{\"enableRetry\":true,\"enableSolutionsButton\":true,\"enableCheckButton\":true,\"type\":\"auto\",\"singlePoint\":false,\"randomAnswers\":true,\"showSolutionsRequiresInput\":true,\"confirmCheckDialog\":false,\"confirmRetryDialog\":false,\"autoCheck\":false,\"passPercentage\":100,\"showScorePoints\":true},\"UI\":{\"checkAnswerButton\":\"Check\",\"submitAnswerButton\":\"Submit\",\"showSolutionButton\":\"Show solution\",\"tryAgainButton\":\"Retry\",\"tipsLabel\":\"Show tip\",\"scoreBarLabel\":\"You got :num out of :total points\",\"tipAvailable\":\"Tip available\",\"feedbackAvailable\":\"Feedback available\",\"readFeedback\":\"Read feedback\",\"wrongAnswer\":\"Wrong answer\",\"correctAnswer\":\"Correct answer\",\"shouldCheck\":\"Should have been checked\",\"shouldNotCheck\":\"Should not have been checked\",\"noInput\":\"Please answer before viewing the solution\",\"a11yCheck\":\"Check the answers. The responses will be marked as correct, incorrect, or unanswered.\",\"a11yShowSolution\":\"Show the solution. The task will be marked with its correct solution.\",\"a11yRetry\":\"Retry the task. Reset all responses and start the task over again.\"},\"confirmCheck\":{\"header\":\"Finish ?\",\"body\":\"Are you sure you wish to finish ?\",\"cancelLabel\":\"Cancel\",\"confirmLabel\":\"Finish\"},\"confirmRetry\":{\"header\":\"Retry ?\",\"body\":\"Are you sure you wish to retry ?\",\"cancelLabel\":\"Cancel\",\"confirmLabel\":\"Confirm\"},\"question\":\"<p>What is the date of succession for Illinois to the Union of the United States and what state number is Illinois?<\\\/p>\\n\"},\"library\":\"H5P.MultiChoice 1.16\",\"metadata\":{\"contentType\":\"Multiple Choice\",\"license\":\"U\",\"title\":\"Untitled Multiple Choice\"},\"subContentId\":\"176046c7-e9f7-4dc0-9740-e2f47ca4a736\"},\"useSeparator\":\"auto\"}]},\"library\":\"H5P.Column 1.15\",\"subContentId\":\"e8c99065-51d4-490a-a11d-e69ffdf921c2\",\"metadata\":{\"contentType\":\"Column\",\"license\":\"U\",\"title\":\"Illinois Statehood\"}}],\"behaviour\":{\"baseColor\":\"#1768c4\",\"defaultTableOfContents\":true,\"progressIndicators\":true,\"progressAuto\":true,\"displaySummary\":true},\"read\":\"Read\",\"displayTOC\":\"Display &#039;Table of contents&#039;\",\"hideTOC\":\"Hide &#039;Table of contents&#039;\",\"nextPage\":\"Next page\",\"previousPage\":\"Previous page\",\"chapterCompleted\":\"Page completed!\",\"partCompleted\":\"@pages of @total completed\",\"incompleteChapter\":\"Incomplete page\",\"navigateToTop\":\"Navigate to the top\",\"markAsFinished\":\"I have finished this page\",\"fullscreen\":\"Fullscreen\",\"exitFullscreen\":\"Exit fullscreen\",\"bookProgressSubtext\":\"@count of @total pages\",\"interactionsProgressSubtext\":\"@count of @total interactions\",\"submitReport\":\"Submit Report\",\"restartLabel\":\"Restart\",\"summaryHeader\":\"Summary\",\"allInteractions\":\"All interactions\",\"unansweredInteractions\":\"Unanswered interactions\",\"scoreText\":\"@score \\\/ @maxscore\",\"leftOutOfTotalCompleted\":\"@left of @max interactions completed\",\"noInteractions\":\"No interactions\",\"score\":\"Score\",\"summaryAndSubmit\":\"Summary &amp; submit\",\"noChapterInteractionBoldText\":\"You have not interacted with any pages.\",\"noChapterInteractionText\":\"You have to interact with at least one page before you can see the summary.\",\"yourAnswersAreSubmittedForReview\":\"Your answers are submitted for review!\",\"bookProgress\":\"Book progress\",\"interactionsProgress\":\"Interactions progress\",\"totalScoreLabel\":\"Total score\",\"a11y\":{\"progress\":\"Page @page of @total.\",\"menu\":\"Toggle navigation menu\"}}","fullScreen":"1","exportUrl":"\/cypresstesttwo\/wp-content\/uploads\/h5p\/exports\/cypress-multiple-choice-3-assessment-from-wp-1.h5p","embedCode":"<iframe src=\"example.com\/wp-admin\/admin-ajax.php?action=h5p_embed&id=1\" width=\":w\" height=\":h\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\" title=\"Cypress_Multiple choice 3_Assessment_from_WP\"><\/iframe>","resizeCode":"<script src=\"example.com\/wp-content\/plugins\/h5p\/h5p-php-library\/js\/h5p-resizer.js\" charset=\"UTF-8\"><\/script>","url":"example.com\/wp-admin\/admin-ajax.php?action=h5p_embed&id=1","title":"Cypress_Multiple choice 3_Assessment_from_WP","displayOptions":{"frame":true,"export":true,"embed":true,"copyright":true,"icon":true,"copy":false},"metadata":{"title":"Cypress_Multiple choice 3_Assessment_from_WP","license":"U"},"contentUserData":[{"state":"{}"}],"scripts":["\/cypresstesttwo\/wp-content\/uploads\/h5p\/cachedassets\/8fd3895124211f96b4e536c5058438e85cb6e2e1.js"],"styles":["\/cypresstesttwo\/wp-content\/uploads\/h5p\/cachedassets\/8fd3895124211f96b4e536c5058438e85cb6e2e1.css"]}}};</script><script id='ce4wp_form_submit-js-extra'>
var ce4wp_form_submit_data = {"siteUrl":"example.com","url":"example.com\/wp-admin\/admin-ajax.php","nonce":"d2a19177fe","listNonce":"089a960f67","activatedNonce":"97b8d10bb6"};
</script>

The startTag event is correctly emitting: <script> as the raw contents, but the text event is sending part of the JS object in the raw contents:

\"library\":\"H5P.MultiChoice 1.16\",\"metadata\":{\"contentType\":\"Multiple Choice\",\"license\":\"U\",\"title\":\"Untitled Multiple Choice\"},\"subContentId\":\"176046c7-e9f7-4dc0-9740-e2f47ca4a736\"},\"useSeparator\":\"auto\"}]},\"library\":\"H5P.Column 1.15\",\"subContentId\":\"e8c99065-51d4-490a-a11d-e69ffdf921c2\",\"metadata\":{\"contentType\":\"Column\",\"license\":\"U\",\"title\":\"Illinois Statehood\"}}],\"behaviour\":{\"baseColor\":\"#1768c4\",\"defaultTableOfContents\":true,\"progressIndicators\":true,\"progressAuto\":true,\"displaySummary\":true},\"read\":\"Read\",\"displayTOC\":\"Display &#039;Table of contents&#039;\",\"hideTOC\":\"Hide &#039;Table of contents&#039;\",\"nextPage\":\"Next page\",\"previousPage\":\"Previous page\",\"chapterCompleted\":\"Page completed!\",\"partCompleted\":\"@pages of @total completed\",\"incompleteChapter\":\"Incomplete page\",\"navigateToTop\":\"Navigate to the top\",\"markAsFinished\":\"I have finished this page\",\"fullscreen\":\"Fullscreen\",\"exitFullscreen\":\"Exit fullscreen\",\"bookProgressSubtext\":\"@count of @total pages\",\"interactionsProgressSubtext\":\"@count of @total interactions\",\"submitReport\":\"Submit Report\",\"restartLabel\":\"Restart\",\"summaryHeader\":\"Summary\",\"allInteractions\":\"All interactions\",\"unansweredInteractions\":\"Unanswered interactions\",\"scoreText\":\"@score \\\/ @maxscore\",\"leftOutOfTotalCompleted\":\"@left of @max interactions completed\",\"noInteractions\":\"No interactions\",\"score\":\"Score\",\"summaryAndSubmit\":\"Summary &amp; submit\",\"noChapterInteractionBoldText\":\"You have not interacted with any pages.\",\"noChapterInteractionText\":\"You have to interact with at least one page before you can see the summary.\",\"yourAnswersAreSubmittedForReview\":\"Your answers are submitted for review!\",\"bookProgress\":\"Book progress\",\"interactionsProgress\":\"Interactions progress\",\"totalScoreLabel\":\"Total score\",\"a11y\":{\"progress\":\"Page @page of @total.\",\"menu\":\"Toggle navigation menu\"}}","fullScreen":"1","exportUrl":"\/cypresstesttwo\/wp-content\/uploads\/h5p\/exports\/cypress-multiple-choice-3-assessment-from-wp-1.h5p","embedCode":"<iframe src=\"example.com\/wp-admin\/admin-ajax.php?action=h5p_embed&id=1\" width=\":w\" height=\":h\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\" title=\"Cypress_Multiple choice 3_Assessment_from_WP\"><\/iframe>","resizeCode":"<script src=\"example.com\/wp-content\/plugins\/h5p\/h5p-php-library\/js\/h5p-resizer.js\" charset=\"UTF-8\"><\/script>","url":"example.com\/wp-admin\/admin-ajax.php?action=h5p_embed&id=1","title":"Cypress_Multiple choice 3_Assessment_from_WP","displayOptions":{"frame":true,"export":true,"embed":true,"copyright":true,"icon":true,"copy":false},"metadata":{"title":"Cypress_Multiple choice 3_Assessment_from_WP","license":"U"},"contentUserData":[{"state":"{}"}],"scripts":["\/cypresstesttwo\/wp-content\/uploads\/h5p\/cachedassets\/8fd3895124211f96b4e536c5058438e85cb6e2e1.js"],"styles":["\/cypresstesttwo\/wp-content\/uploads\/h5p\/cachedassets\/8fd3895124211f96b4e536c5058438e85cb6e2e1.css"]}}};

I expect the raw contents of the text event to match the full contents.

fb55 commented 1 year ago

This is likely caused by a breaking change in 7.0.0 (see https://github.com/inikulin/parse5/releases/tag/v7.0.0), which will now emit multiple text events for long sections (where we drop the underlying buffer). This is a bug fix for a long standing issue within parse5, where the raw text was not reliable.

Hope this makes sense!

fb55 commented 1 year ago

This is the relevant PR: https://github.com/inikulin/parse5/pull/432

emilyrohrbough commented 1 year ago

@fb55 I wasn't observing multiple test events for long sections. Only the last portion of the text was being sent. Is that expected at all?

fb55 commented 1 year ago

Looking at the code in the first post of this issue, the important thing to note is that the data event will only fire for tokens that have not been captured before. So if you assign a listener for text, this will lead to the data event not receiving any data from text tokens (unless emitText is called explicitly). With the listeners removed, I receive the full input as the output.

Looking at cypress' source code, this function doesn't deal with multiple text events for script tags (the relevant logic should be moved to endTag): https://github.com/cypress-io/cypress/blob/af0da61b8cf07ef260cf50587ba58b1453e21cdc/packages/rewriter/lib/html-rules.ts#L45-L55

emilyrohrbough commented 1 year ago

@fb55 Sorry for the late replay. THANK YOU for taking a deeper look into this. I will give this a shot and see if this resolves the issue.