pjhill commented 3 years ago

Description

💡 We might be able to shave some time off of Cypress tests by using an ES Build preprocessor instead of webpack.

https://github.com/bahmutov/cypress-esbuild-preprocessor

That's pretty strong statement: the ESBuild preprocessor is 17 times faster in bundling the support file and 50 (fifty!) times faster in bundling the spec file. - https://glebbahmutov.com/blog/fast-esbuild/

Tasks

[x] Measure baseline Cypress Webpack preprocessor times - done
[x] Reference the aforementioned blog post describing the process of swapping out the default build preprocessor in Cypress for a new one in order to configure vets-website similarly - done
[x] Measure improved Cypress ESBuild preprocessor times - done
[x] Debug issues with unexpected < error messages being exhibited in some specs - 3 points
[x] Debug issues with injectAxe reference error messages being exhibited in some specs - 3 points
[x] Debug issues with The following error originated from your test code, not from Cypress error messages being exhibited in at least one spec - 3 points

Acceptance Criteria

[ ] Cypress tests continue to function as expected
[ ] Cypress test flakiness has not increased above current levels
[ ] Cypress test execution time has been reduced

aubreyarcangel commented 3 years ago

ESbuild is running, but only a few tests are compiling. Below is a WIP list of specs with their errors

Test	Error
src/applications/appeals/10182/tests/10182-notice-of-disagreement.cypress.spec.js	Build failed with 3 errors: src/applications/appeals/10182/content/boardReview.js:6:4: error: Unexpected "<" src/applications/appeals/10182/content/representative.js:7:2: error: Unexpected "<" src/platform/forms/preSubmitInfo.js:6:4: error: Unexpected "<"
src/applications/ask-a-question/tests/cypress/ask-a-question.authed.cypress.spec.js	Build failed with 1 error: src/applications/ask-a-question/form/form.js:85:4: error: Unexpected "<"
src/applications/ask-a-question/tests/cypress/ask-a-question.unauthed.cypress.spec.js	Build failed with 1 error: src/applications/ask-a-question/form/form.js:85:4: error: Unexpected "<"
src/applications/caregivers/tests/e2e/1010cg.cypress.spec.js	Build failed with 2 errors: src/applications/caregivers/definitions/UIDefinitions/veteranUI.js:27:4: error: Unexpected "<" src/applications/caregivers/definitions/content.js:56:2: error: Unexpected "<"
src/applications/coronavirus-research/tests/reviewAndSubmit.cypress.spec.js	require is not defined Because this error occurred during a before all hook we are skipping the remaining tests in the current suite: when entering valid informa...
src/applications/coronavirus-vaccination-expansion/tests/coronavirus-vaccination-expansion.cypress.spec.js	src/applications/coronavirus-vaccination-expansion/config/PreSubmitinfo.js:6:10: error: Unexpected "<" src/applications/coronavirus-vaccination-expansion/config/address/addressInformation.js:42:8: error: Unexpected "<" src/applications/coronavirus-vaccination-expansion/config/attestation/helpers.js:5:21: error: Unexpected "<" src/applications/coronavirus-vaccination-expansion/config/personal-information/personal-information.js:30:8: error: Unexpected "<" src/applications/coronavirus-vaccination-expansion/config/va-location/va-location.js:6:9: error: Unexpected "<"
src/applications/coronavirus-vaccination/tests/e2e/hideauth.cypress.spec.js	injectAxe (xhr) GET --- /v0/maintenance_windows/ ReferenceError require is not defined Because this error occurred during a before all hook we are skipping the remaining tests in the current suite: when entering app with auth...
src/applications/coronavirus-vaccination/tests/e2e/signup.cypress.spec.js	injectAxe ReferenceError require is not defined Because this error occurred during a before all hook we are skipping the remaining tests in the current suite: when entering valid contact...
	injectAxe (xhr stub) PUT 200 /covid_vaccine/v0/registration/opt_out ReferenceError require is not defined View stack trace
src/applications/coronavirus-vaccination/tests/e2e/unsubscribeFail.cypress.spec.js	injectAxe (xhr) PUT --- /covid_vaccine/v0/registration/opt_out ReferenceError require is not defined
src/applications/debt-letters/tests/e2e/debt-letters.cypress.spec.js	none
src/applications/debt-letters/tests/e2e/diary-codes-content.cypress.spec.js	none
src/applications/debt-letters/tests/e2e/diary-codes-next-step.cypress.spec.js	none
src/applications/disability-benefits/2346/tests/mdot.cypress.spec.js	Build failed with 1 error: src/applications/disability-benefits/2346/schemas/address-schema.js:58:2: error: Unexpected "<"
src/applications/disability-benefits/686c-674/tests/e2e/686C-674-ancilliary.cypress.spec.js	Error: Build failed with 13 errors: src/applications/disability-benefits/686c-674/components/CustomPreSubmitInfo.js:6:4: error: Unexpected "<" src/applications/disability-benefits/686c-674/config/address-schema.js:61:2: error: Unexpected "<" src/applications/disability-benefits/686c-674/config/chapters/674/student-income-information/studentIncomeInformation.js:18:6: error: Unexpected "<" src/applications/disability-benefits/686c-674/config/chapters/674/student-last-term-information/studentLastTerm.js:20:6: error: Unexpected "<" src/applications/disability-benefits/686c-674/config/chapters/674/student-networth-information/studentNetworthInformation.js:18:6: error: Unexpected "<" ...
src/applications/disability-benefits/686c-674/tests/e2e/686C-674.cypress.spec.js	Error: Build failed with 13 errors: src/applications/disability-benefits/686c-674/components/CustomPreSubmitInfo.js:6:4: error: Unexpected "<" src/applications/disability-benefits/686c-674/config/address-schema.js:61:2: error: Unexpected "<" src/applications/disability-benefits/686c-674/config/chapters/674/student-income-information/studentIncomeInformation.js:18:6: error: Unexpected "<" src/applications/disability-benefits/686c-674/config/chapters/674/student-last-term-information/studentLastTerm.js:20:6: error: Unexpected "<" src/applications/disability-benefits/686c-674/config/chapters/674/student-networth-information/studentNetworthInformation.js:18:6: error: Unexpected "<" ...
src/applications/disability-benefits/996/tests/hlr-wizard.cypress.spec.js	injectAxe ReferenceError require is not defined Because this error occurred during a before each hook we are skipping the remaining tests in the current suite: HLR wizard
src/applications/disability-benefits/996/tests/hlr.cypress.spec.js	Build failed with 3 errors: src/applications/static-pages/wizard/index.js:133:6: error: Unexpected "<" src/platform/forms/preSubmitInfo.js:6:4: error: Unexpected "<" src/platform/site-wide/cta-widget/index.js:112:8: error: Unexpected "<"
src/applications/disability-benefits/all-claims/tests/all-claims-wizard.cypress.spec.js	injectAxe ReferenceError require is not defined Because this error occurred during a before each hook we are skipping the remaining tests in the current suite: 526 wizard
src/applications/disability-benefits/all-claims/tests/all-claims.cypress.spec.js	Build failed with 9 errors: src/applications/disability-benefits/all-claims/pages/additionalBehaviorChanges.js:6:2: error: Unexpected "<" src/applications/disability-benefits/all-claims/pages/contactInformation.js:92:8: error: Unexpected "<" src/applications/disability-benefits/all-claims/pages/incidentDescription.js:7:2: error: Unexpected "<" src/applications/disability-benefits/all-claims/pages/secondaryIncidentDescription.js:7:2: error: Unexpected "<" src/applications/disability-benefits/all-claims/pages/secondaryOtherSources.js:11:4: error: Unexpected "<" ...
src/applications/disability-benefits/view-payments/tests/e2e/view-payments.cypress.spec.js	injectAxe ReferenceError require is not defined
src/applications/edu-benefits/0994/tests/e2e/edu-0994.cypress.spec.js	Build failed with 3 errors: src/applications/static-pages/wizard/index.js:133:6: error: Unexpected "<" src/platform/forms/preSubmitInfo.js:6:4: error: Unexpected "<" src/platform/site-wide/cta-widget/index.js:112:8: error: Unexpected "<"
src/applications/edu-benefits/10203/tests/edu-10203.cypress.spec.js	Build failed with 1 error: src/platform/forms/preSubmitInfo.js:6:4: error: Unexpected "<"
src/applications/edu-benefits/1990/tests/e2e/edu-1990.cypress.spec.js	Build failed with 2 errors: src/applications/static-pages/wizard/index.js:133:6: error: Unexpected "<" src/platform/forms/preSubmitInfo.js:6:4: error: Unexpected "<"
src/applications/edu-benefits/1990e/tests/e2e/edu-1990e.cypress.spec.js	Build failed with 2 errors: src/applications/static-pages/wizard/index.js:133:6: error: Unexpected "<" src/platform/forms/preSubmitInfo.js:6:4: error: Unexpected "<"
src/applications/edu-benefits/1990n/tests/e2e/edu-1990n.cypress.spec.js	Build failed with 2 errors: src/applications/static-pages/wizard/index.js:133:6: error: Unexpected "<" src/platform/forms/preSubmitInfo.js:6:4: error: Unexpected "<"
src/applications/edu-benefits/1995/tests/e2e/edu-1995.cypress.spec.js	Build failed with 2 errors: src/applications/static-pages/wizard/index.js:133:6: error: Unexpected "<" src/platform/forms/preSubmitInfo.js:6:4: error: Unexpected "<"
src/applications/edu-benefits/5490/tests/e2e/edu-5490.cypress.spec.js	Build failed with 2 errors: src/applications/static-pages/wizard/index.js:133:6: error: Unexpected "<" src/platform/forms/preSubmitInfo.js:6:4: error: Unexpected "<"
src/applications/edu-benefits/5495/tests/e2e/edu-5495.cypress.spec.js	Build failed with 2 errors: src/applications/static-pages/wizard/index.js:133:6: error: Unexpected "<" src/platform/forms/preSubmitInfo.js:6:4: error: Unexpected "<"
src/applications/facility-locator/tests/e2e/accessibility.cypress.spec.js	The following error originated from your test code, not from Cypress. > BUILDTYPE is not defined When Cypress detects uncaught errors originating from your test code it will automatically fail the current test. Cypress could not associate this error to any specific test. We dynamically generated a new test to display this failure. Check your console for the stack trace or click this message to see where it originated from.

aubreyarcangel commented 3 years ago

Baseline measurement Average time for spec build with webpack: 3-7 seconds

Although I'm hitting build errors with most of the specs, the few specs that were able to build using ESbuild: less than 1 second

aubreyarcangel commented 3 years ago

I believe the source of the the Unexpected "<" is because esbuild is not parsing jsx.

When converting

  errorText: () => {
     <p>
       If it still doesn’t work, please <CallMyVA311 />
     </p>

to

  errorText: () => {
    React.createElement('p', null, 'If it still doesn\u2019t work, please ');
  },

The error goes away.

aubreyarcangel commented 3 years ago

I have about 80% of tests running.

All specs that use form-tester have this error message:

> BUILDTYPE is not defined

aubreyarcangel commented 3 years ago

The injectAxe error require is not defined is coming from cypress-axe

https://github.com/component-driven/cypress-axe/pull/86

The latest version 0.12.2 is supposed to fix this error, which we upgraded to two months, but it's still giving off the error using esbuild.

Using this workaround works: https://github.com/component-driven/cypress-axe/issues/82#issuecomment-779883508

aubreyarcangel commented 3 years ago

Updated inventory of spec errors:

144 total tests

56 successful run
40 cy.visit() 404 errors
27 __dirname is not defined (node path import issue)
5 partially run but receive cy.visit() 404 error
5 Build fail react-router related
4 global is not defined

Test	Error
src/applications/appeals/10182/tests/10182-notice-of-disagreement.cypress.spec.js	> __dirname is not defined
src/applications/ask-a-question/tests/cypress/ask-a-question.authed.cypress.spec.js	> __dirname is not defined
src/applications/ask-a-question/tests/cypress/ask-a-question.unauthed.cypress.spec.js	> __dirname is not defined
src/applications/caregivers/tests/e2e/1010cg.cypress.spec.js	> __dirname is not defined
src/applications/coronavirus-research/tests/reviewAndSubmit.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/coronavirus-research/volunteer/sign-up The response we received from your web server was: > 404: Not Found This was considered a failure because the status code was not 2xx. If you do not want status codes to cause failures pass the option: failOnStatusCode: false Because this error occurred during a before all hook we are skipping the remaining tests in the current suite: when entering valid informa...
src/applications/coronavirus-vaccination-expansion/tests/coronavirus-vaccination-expansion.cypress.spec.js	successful
src/applications/coronavirus-vaccination-expansion/tests/coronavirus-vaccination-expansion-form-validation.cypress.spec.js	successful
src/applications/coronavirus-vaccination-expansion/tests/coronavirus-vaccination-expansion.cypress.spec.js	> __dirname is not defined
src/applications/coronavirus-vaccination/tests/e2e/hideauth.cypress.spec.js	No elements found for include in frame Context
src/applications/coronavirus-vaccination/tests/e2e/signup.cypress.spec.js	successful
src/applications/coronavirus-vaccination/tests/e2e/unsubscribe.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/covid-19-vaccine/stay-informed/unsubscribe?sid=12345 The response we received from your web server was: > 404: Not Found This was considered a failure because the status code was not 2xx. If you do not want status codes to cause failures pass the option: failOnStatusCode: false
src/applications/coronavirus-vaccination/tests/e2e/unsubscribeFail.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/covid-19-vaccine/stay-informed/unsubscribe?sid=12345 The response we received from your web server was: > 404: Not Found This was considered a failure because the status code was not 2xx. If you do not want status codes to cause failures pass the option: failOnStatusCode: false
src/applications/debt-letters/tests/e2e/debt-letters.cypress.spec.js	does not run
src/applications/debt-letters/tests/e2e/diary-codes-content.cypress.spec.js	does not run
src/applications/debt-letters/tests/e2e/diary-codes-next-step.cypress.spec.js	does not run
src/applications/disability-benefits/2346/tests/mdot.cypress.spec.js
src/applications/disability-benefits/686c-674/tests/e2e/686C-674-ancilliary.cypress.spec.js	> __dirname is not defined
src/applications/disability-benefits/686c-674/tests/e2e/686C-674.cypress.spec.js	> __dirname is not defined
src/applications/disability-benefits/996/tests/hlr-wizard.cypress.spec.js	successful
src/applications/disability-benefits/996/tests/hlr.cypress.spec.js	> global is not defined
src/applications/disability-benefits/all-claims/tests/all-claims-wizard.cypress.spec.js	successful
src/applications/disability-benefits/all-claims/tests/all-claims.cypress.spec.js	successful
src/applications/disability-benefits/view-payments/tests/e2e/view-payments.cypress.spec.js	successful
src/applications/discharge-wizard/tests/e2e/discharge-wizard.cypress.spec.js	successful
src/applications/edu-benefits/0994/tests/e2e/edu-0994.cypress.spec.js	> global is not defined
src/applications/edu-benefits/10203/tests/edu-10203.cypress.spec.js	> __dirname is not defined
src/applications/edu-benefits/1990/tests/e2e/edu-1990.cypress.spec.js	> manifest_default.join is not a function
src/applications/edu-benefits/1990e/tests/e2e/edu-1990e.cypress.spec.js	> __dirname is not defined
src/applications/edu-benefits/1990n/tests/e2e/edu-1990n.cypress.spec.js	> __dirname is not defined
src/applications/edu-benefits/1990s/tests/vrrap.cypress.spec.js	> global is not defined
src/applications/edu-benefits/1995/tests/e2e/edu-1995.cypress.spec.js	> __dirname is not defined
src/applications/edu-benefits/5490/tests/e2e/edu-5490.cypress.spec.js	> __dirname is not defined
src/applications/edu-benefits/5495/tests/e2e/edu-5495.cypress.spec.js	> __dirname is not defined
src/applications/edu-benefits/tests/edu-apply-wizard.cypress.spec.js	> process is not defined
src/applications/facility-locator/tests/e2e/accessibility.cypress.spec.js	__dirname is not defined
src/applications/facility-locator/tests/e2e/ccpDisabled.cypress.spec.js	__dirname is not defined
src/applications/facility-locator/tests/e2e/ccpEnabled.cypress.spec.js	__dirname is not defined
src/applications/facility-locator/tests/e2e/detailPage.cypress.spec.js	__dirname is not defined
src/applications/facility-locator/tests/e2e/facilitySearch.cypress.spec.js	__dirname is not defined
src/applications/facility-locator/tests/e2e/gaEvents.cypress.spec.js	__dirname is not defined
src/applications/facility-locator/tests/e2e/map-zoom.cypress.spec.js	successful
src/applications/facility-locator/tests/e2e/mobile.cypress.spec.js	__dirname is not defined
src/applications/facility-locator/tests/e2e/providerSearch.cypress.spec.js	__dirname is not defined
src/applications/financial-status-report/tests/e2e/fsr-5655-wizard.cypress.spec.js	successful
src/applications/financial-status-report/tests/e2e/fsr-5655.cypress.spec.js	__dirname is not defined
src/applications/find-forms/tests/e2e/find-forms-required.cypress.spec.js	successful
src/applications/gi/tests/e2e/00-required.cypress.spec.js	Error: Build failed with 13 errors: node_modules/react-router-dom/esm/react-router-dom.js:1:17: error: No matching export in "node_modules/react-router/es/index.js" for import "__RouterContext" node_modules/react-router-dom/esm/react-router-dom.js:1:34: error: No matching export in "node_modules/react-router/es/index.js" for import "matchPath" node_modules/react-router-dom/esm/react-router-dom.js:2:9: error: No matching export in "node_modules/react-router/es/index.js" for import "MemoryRouter" node_modules/react-router-dom/esm/react-router-dom.js:2:23: error: No matching export in "node_modules/react-router/es/index.js" for import "Prompt" node_modules/react-router-dom/esm/react-router-dom.js:2:56: error: No matching export in "node_modules/react-router/es/index.js" for import "StaticRouter"
src/applications/gi/tests/e2e/02-profile-error.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/gi-bill-comparison-tool/profile/99999999 The response we received from your web server was: > 404: Not Found
src/applications/gi/tests/e2e/03-vet-tec.cypress.spec.js	Error: Build failed with 13 errors: node_modules/react-router-dom/esm/react-router-dom.js:1:17: error: No matching export in "node_modules/react-router/es/index.js" for import "__RouterContext" node_modules/react-router-dom/esm/react-router-dom.js:1:34: error: No matching export in "node_modules/react-router/es/index.js" for import "matchPath" node_modules/react-router-dom/esm/react-router-dom.js:2:9: error: No matching export in "node_modules/react-router/es/index.js" for import "MemoryRouter" node_modules/react-router-dom/esm/react-router-dom.js:2:23: error: No matching export in "node_modules/react-router/es/index.js" for import "Prompt" node_modules/react-router-dom/esm/react-router-dom.js:2:56: error: No matching export in "node_modules/react-router/es/index.js" for import "StaticRouter"
src/applications/gi/tests/e2e/04-dea.cypress.spec.js	Error: Build failed with 13 errors: node_modules/react-router-dom/esm/react-router-dom.js:1:17: error: No matching export in "node_modules/react-router/es/index.js" for import "__RouterContext" node_modules/react-router-dom/esm/react-router-dom.js:1:34: error: No matching export in "node_modules/react-router/es/index.js" for import "matchPath" node_modules/react-router-dom/esm/react-router-dom.js:2:9: error: No matching export in "node_modules/react-router/es/index.js" for import "MemoryRouter" node_modules/react-router-dom/esm/react-router-dom.js:2:23: error: No matching export in "node_modules/react-router/es/index.js" for import "Prompt" node_modules/react-router-dom/esm/react-router-dom.js:2:56: error: No matching export in "node_modules/react-router/es/index.js" for import "StaticRouter"
src/applications/gi/tests/e2e/05-ojt.cypress.spec.js	Error: Build failed with 13 errors: node_modules/react-router-dom/esm/react-router-dom.js:1:17: error: No matching export in "node_modules/react-router/es/index.js" for import "__RouterContext" node_modules/react-router-dom/esm/react-router-dom.js:1:34: error: No matching export in "node_modules/react-router/es/index.js" for import "matchPath" node_modules/react-router-dom/esm/react-router-dom.js:2:9: error: No matching export in "node_modules/react-router/es/index.js" for import "MemoryRouter" node_modules/react-router-dom/esm/react-router-dom.js:2:23: error: No matching export in "node_modules/react-router/es/index.js" for import "Prompt" node_modules/react-router-dom/esm/react-router-dom.js:2:56: error: No matching export in "node_modules/react-router/es/index.js" for import "StaticRouter"
src/applications/gi/tests/e2e/06-vet-tec-mobile.cypress.spec.js	Error: Build failed with 13 errors: node_modules/react-router-dom/esm/react-router-dom.js:1:17: error: No matching export in "node_modules/react-router/es/index.js" for import "__RouterContext" node_modules/react-router-dom/esm/react-router-dom.js:1:34: error: No matching export in "node_modules/react-router/es/index.js" for import "matchPath" node_modules/react-router-dom/esm/react-router-dom.js:2:9: error: No matching export in "node_modules/react-router/es/index.js" for import "MemoryRouter" node_modules/react-router-dom/esm/react-router-dom.js:2:23: error: No matching export in "node_modules/react-router/es/index.js" for import "Prompt" node_modules/react-router-dom/esm/react-router-dom.js:2:56: error: No matching export in "node_modules/react-router/es/index.js" for import "StaticRouter"
src/applications/gi/tests/e2e/06-vet-tec-mobile.cypress.spec.js	successful
src/applications/hca/tests/hca.cypress.spec.js	> __dirname is not defined
src/applications/health-care-questionnaire/list/tests/e2e/print/failed.print.cypress.spec.js	successful
src/applications/health-care-questionnaire/list/tests/e2e/print/successful.print.cypress.spec.js	successful
src/applications/health-care-questionnaire/list/tests/e2e/00.home.cypress.spec.js	successful
src/applications/health-care-questionnaire/list/tests/e2e/01.tabs.switch.cypress.spec.js	successful
src/applications/health-care-questionnaire/list/tests/e2e/02.bread.crumbs.cypress.spec.js	doesnt run
src/applications/health-care-questionnaire/list/tests/e2e/03.is.downtime.notifier.cypress.spec.js	successful
src/applications/health-care-questionnaire/list/tests/e2e/03.session.storage.is.clearing.cypress.spec.js	Timed out retrying after 4000ms: Expected to find element: #va-breadcrumbs-list > :nth-child(3) > a, but never found it.
src/applications/health-care-questionnaire/list/tests/e2e/03.session.storage.is.populated.cypress.spec.js	successful
src/applications/health-care-questionnaire/list/tests/e2e/03.session.storage.is.populated.cypress.spec.js	successful
src/applications/health-care-questionnaire/list/tests/e2e/04.questionnaire.happy.path.cypress.spec.js	doesnt run
src/applications/health-care-questionnaire/questionnaire/tests/e2e/bugs/questionnaire.unauth.loading.bug.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/health-questionnaires/questionnaires/answer-questions?id=195bc02c0518870fc6b1e302cfc326b61 The response we received from your web server was: > 404: Not Found
src/applications/health-care-questionnaire/questionnaire/tests/e2e/bugs/reason.for.visit.bug.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/health-questionnaires/questionnaires/answer-questions?id=I2-3PYJBEU2DIBW5RZT2XI3PASYGM7YYRD5TFQCLHQXK6YBXREQK5VQ0005&skip The response we received from your web server was:
src/applications/health-care-questionnaire/questionnaire/tests/e2e/bugs/unauthed.no-id.form.should.redirect.disabled.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/health-questionnaires/questionnaires/answer-questions The response we received from your web server was: > 404: Not Found
src/applications/health-care-questionnaire/questionnaire/tests/e2e/bugs/unauthed.no-id.form.should.redirect.enabled.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/health-questionnaires/questionnaires/answer-questions The response we received from your web server was: > 404: Not Found
src/applications/health-care-questionnaire/questionnaire/tests/e2e/content/health.care.questionniare.footer.loads.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/health-questionnaires/questionnaires/answer-questions?id=I2-3PYJBEU2DIBW5RZT2XI3PASYGM7YYRD5TFQCLHQXK6YBXREQK5VQ0005 The response we received from your web server was: > 404: Not Found
src/applications/health-care-questionnaire/questionnaire/tests/e2e/required-fields/reason.for.visit.is.required.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/health-questionnaires/questionnaires/answer-questions?id=I2-3PYJBEU2DIBW5RZT2XI3PASYGM7YYRD5TFQCLHQXK6YBXREQK5VQ0005&skip The response we received from your web server was: > 404: Not Found
src/applications/health-care-questionnaire/questionnaire/tests/e2e/00.questionnaire.is.disabled.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/health-questionnaires/questionnaires/answer-questions?id=195bc02c0518870fc6b1e302cfc326b61 The response we received from your web server was: > 404: Not Found
src/applications/health-care-questionnaire/questionnaire/tests/e2e/00.questionnaire.loads.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/health-questionnaires/questionnaires/answer-questions?id=I2-3PYJBEU2DIBW5RZT2XI3PASYGM7YYRD5TFQCLHQXK6YBXREQK5VQ0005 The response we received from your web server was: > 404: Not Found
src/applications/health-care-questionnaire/questionnaire/tests/e2e/01.questionnaire.demographics.address.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/health-questionnaires/questionnaires/answer-questions/veteran-information?id=I2-3PYJBEU2DIBW5RZT2XI3PASYGM7YYRD5TFQCLHQXK6YBXREQK5VQ0005&skip The response we received from your web server was: > 404: Not Found
src/applications/health-care-questionnaire/questionnaire/tests/e2e/01.questionnaire.demographics.basicInformation.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/health-questionnaires/questionnaires/answer-questions/veteran-information?id=I2-3PYJBEU2DIBW5RZT2XI3PASYGM7YYRD5TFQCLHQXK6YBXREQK5VQ0005&skip The response we received from your web server was: > 404: Not Found
src/applications/health-care-questionnaire/questionnaire/tests/e2e/01.questionnaire.demographics.phoneNumbers.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/health-questionnaires/questionnaires/answer-questions/veteran-information?id=I2-3PYJBEU2DIBW5RZT2XI3PASYGM7YYRD5TFQCLHQXK6YBXREQK5VQ0005&skip The response we received from your web server was: > 404: Not Found
src/applications/health-care-questionnaire/questionnaire/tests/e2e/02.questionnaire.form.happy.path.cypress.spec.js	> global is not defined
src/applications/health-care-questionnaire/questionnaire/tests/e2e/03.questionnaire.landing.page.accessibility.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/health-questionnaires/questionnaires/answer-questions/veteran-information?id=I2-3PYJBEU2DIBW5RZT2XI3PASYGM7YYRD5TFQCLHQXK6YBXREQK5VQ0005&skip The response we received from your web server was: > 404: Not Found
src/applications/health-care-questionnaire/questionnaire/tests/e2e/04.appointment.id.is.missing.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/health-questionnaires/questionnaires/answer-questions/introduction The response we received from your web server was: > 404: Not Found
src/applications/health-care-questionnaire/questionnaire/tests/e2e/04.appointment.id.is.present.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/health-questionnaires/questionnaires/answer-questions?id=I2-3PYJBEU2DIBW5RZT2XI3PASYGM7YYRD5TFQCLHQXK6YBXREQK5VQ0005 The response we received from your web server was: > 404: Not Found
src/applications/health-care-questionnaire/questionnaire/tests/e2e/08.appointment.is.in.past.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/health-questionnaires/questionnaires/answer-questions?id=I2-3PYJBEU2DIBW5RZT2XI3PASYGM7YYRD5TFQCLHQXK6YBXREQK5VQ0005 The response we received from your web server was: > 404: Not Found
src/applications/lgy/coe/tests/coe.cypress.spec.js	> __dirname is not defined
src/applications/messages/tests/messages.cypress.spec.js	successful
src/applications/messages/tests/messages.cypress.spec.js	successful
src/applications/personalization/dashboard-2/tests/e2e/benefits-of-interest.cypress.spec.js	successful
src/applications/personalization/dashboard-2/tests/e2e/benefits-of-interest.cypress.spec.js	successful
src/applications/personalization/dashboard-2/tests/e2e/claims-and-appeals.cypress.spec.js	successful
src/applications/personalization/dashboard-2/tests/e2e/claims-and-appeals.cypress.spec.js	successful
src/applications/personalization/dashboard-2/tests/e2e/in-progress-forms.cypress.spec.js	successful
src/applications/personalization/dashboard-2/tests/e2e/loa1.cypress.spec.js	successful
src/applications/personalization/dashboard-2/tests/e2e/loa3.cypress.spec.js	successful
src/applications/personalization/dashboard-2/tests/e2e/messaging-error.cypress.spec.js	successful
src/applications/personalization/dashboard-2/tests/e2e/mpi-connection-error.cypress.spec.js	successful
src/applications/personalization/dashboard-2/tests/e2e/not-in-mpi-error.cypress.spec.js	successful
src/applications/personalization/dashboard/tests/e2e/dashboard.all-widgets.cypress.spec.js	successful
src/applications/personalization/dashboard/tests/e2e/dashboard.find-benefits-empty.cypress.spec.js	successful
src/applications/personalization/dashboard/tests/e2e/dashboard.find-benefits-selected.cypress.spec.js	successful
src/applications/personalization/dashboard/tests/e2e/dashboard.health-care-widgets.cypress.spec.js	successful
src/applications/personalization/dashboard/tests/e2e/dashboard.minimal-widgets.cypress.spec.js	successful
src/applications/personalization/profile/tests/e2e/address-validation/bad-unit.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/personal-information The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/address-validation/confirm-address.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/personal-information The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/address-validation/confirm-state-diff.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/personal-information The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/address-validation/edit-after-validation.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/personal-information The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/address-validation/international-address.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/personal-information The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/address-validation/low-confidence.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/personal-information The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/address-validation/military-address.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/personal-information The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/address-validation/missing-unit.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/personal-information The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/address-validation/no-changes-made.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/personal-information The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/address-validation/one-suggestion.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/personal-information The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/address-validation/two-suggestions.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/personal-information The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/address-validation/valid-address.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/personal-information The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/direct-deposit/gating.cypress.spec.js	Runs but hitting 404 cy.visit() error cy.visit() failed trying to load: http://localhost:3002/profile/direct-deposit The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/direct-deposit/non-2fa.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/direct-deposit The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/direct-deposit/update-flow.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/direct-deposit The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/personal-and-contact-info/fields-check.cypress.spec.js	successful
src/applications/personalization/profile/tests/e2e/personal-and-contact-info/mailing-address.cypress.spec.js	successful
src/applications/personalization/profile/tests/e2e/personal-and-contact-info/modals.cypress.spec.js	successful
src/applications/personalization/profile/tests/e2e/personal-and-contact-info/vap-error.cypress.spec.js	successful
src/applications/personalization/profile/tests/e2e/loa1.cypress.spec.js	Runs but gets 404 error cy.visit() failed trying to load: http://localhost:3002/profile/connected-applications The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/nametag.cypress.spec.js	successful
src/applications/personalization/profile/tests/e2e/profile-a11y.cypress.spec.js	successful
src/applications/personalization/profile/tests/e2e/profile.connected-apps.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/connected-applications The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/profile.mpi-error.cypress.spec.js	Runs, but gets 404 error cy.visit() failed trying to load: http://localhost:3002/profile/connected-applications The response we received from your web server was: > 404: Not Found
src/applications/personalization/profile/tests/e2e/profile.not-in-mpi-error.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/profile/connected-applications The response we received from your web server was: > 404: Not Found
src/applications/personalization/rated-disabilities/tests/e2e/rated-disabilities.cypress.spec.js	successful
src/applications/personalization/view-dependents/tests/e2e/view-dependents.cypress.spec.js	successful
src/applications/personalization/view-representative/tests/view-representative.cypress.spec.js	Runs, but displays error: No elements found for include in frame Context
src/applications/search/tests/e2e/00-required.cypress.spec.js	successful
src/applications/static-pages/dependency-verification/tests/dependency-verification.cypress.spec.js	successful
src/applications/static-pages/health-care-manage-benefits/schedule-view-va-appointments-page/tests/index.cypress.spec.js	successful
src/applications/vaos/tests/e2e/appointment-list.cypress.spec.js	successful
src/applications/vaos/tests/e2e/community-care.cypress.spec.js	cy.visit() failed trying to load: http://localhost:3002/health-care/schedule-view-va-appointments/appointments/new-appointment/ The response we received from your web server was: > 404: Not Found
src/applications/vaos/tests/e2e/covid19-vaccine.cypress.spec.js	Runs, but receives 404 error towards the end of spec cy.visit() failed trying to load: http://localhost:3002/health-care/schedule-view-va-appointments/appointments/new-covid-19-vaccine-booking The response we received from your web server was: > 404: Not Found
src/applications/vaos/tests/e2e/va-appointment.cypress.spec.js	successful
src/applications/vaos/tests/e2e/va-request.cypress.spec.js	successful
src/applications/virtual-agent/tests/virtual-agent.cypress.spec.js	successful
src/applications/vre/28-1900/tests/e2e/chapter31-loa1.cypress.spec.js	successful
src/applications/vre/28-1900/tests/e2e/chapter31-maximal.cypress.spec.js	> __dirname is not defined
src/applications/vre/28-1900/tests/e2e/chapter31-wizard.cypress.spec.js	successful
src/applications/vre/28-8832/tests/e2e/dependent-workflow.cypress.spec.js	> __dirname is not defined
src/applications/vre/28-8832/tests/e2e/veteran-workflow-not-logged-in.cypress.spec.js	> __dirname is not defined
src/applications/yellow-ribbon/e2e/tests/yellow-ribbon.cypress.spec.js	successful
src/platform/site-wide/side-nav/tests/e2e/sideNav.cypress.spec.js	successful
src/platform/site-wide/user-nav/tests/e2e/00-required.cypress.spec.js	successful

aubreyarcangel commented 3 years ago

List of remaining tasks for esbuild swap:

Resolve remaining cy.visit() 404 errors
- 40 specs with this error but may be greatly reduced after getting content build running
Resolve __dirname not defined
- 27 specs with this error
- Connect with Eugene to make sense of webpack options for esbuild
Resolve react-router build fail
- 5 specs with this error
- guess: related to imports
Resolve global is not defined
- 5 specs with this error
Update CI scripts to run cypress esbuild commands
- jenkins scripts
- docker scripts
- circleCI

aubreyarcangel commented 3 years ago

In reference to the __dirname is not defined error, this is a known issue within esbuild: https://github.com/evanw/esbuild/issues/859

aubreyarcangel commented 3 years ago

Status update:

103/144 tests passing.

Remaining task list:

Resolve __dirname not defined
- 27 specs with this error
- Connect with Eugene to make sense of webpack options for esbuild
Resolve react-router build fail
- 5 specs with this error
- guess: related to imports
Resolve global is not defined
- 5 specs with this error
Update CI scripts to run cypress esbuild commands
- jenkins scripts
- docker scripts
- circleCI

aubreyarcangel commented 3 years ago

List of remaining tasks for esbuild swap:

132/146 tests passing

[x] Resolve remaining cy.visit() 404 errors
- 40 specs with this error but may be greatly reduced after getting content build running
[x] Resolve __dirname not defined
- 27 specs with this error
- Connect with Eugene to make sense of webpack options for esbuild
[ ] Resolve react-router build fail
- 5 specs with this error
- guess: related to imports
[ ] Resolve global is not defined
- 5 specs with this error
[ ] Resolve test that do not run
- 5 specs with this error
[ ] Update CI scripts to run cypress esbuild commands
- jenkins scripts
- docker scripts
- circleCI

aubreyarcangel commented 3 years ago

dropping this here for > global is not defined error

and > process is not defined

https://github.com/evanw/esbuild/blob/master/internal/runtime/runtime.go

fix released 1.5 weeks ago https://github.com/evanw/esbuild/pull/1291

I updated esbuild version to 0.12.3 but still hitting the error

aubreyarcangel commented 3 years ago

Receiving the same react-router error for these specs:

src/applications/gi/tests/e2e/00-required.cypress.spec.js src/applications/gi/tests/e2e/02-profile-error.cypress.spec.js src/applications/gi/tests/e2e/03-vet-tec.cypress.spec.js src/applications/gi/tests/e2e/04-dea.cypress.spec.js src/applications/gi/tests/e2e/05-ojt.cypress.spec.js src/applications/gi/tests/e2e/06-vet-tec-mobile.cypress.spec.js src/applications/gi/tests/e2e/06-vet-tec-mobile.cypress.spec.js

Error: Build failed with 13 errors:
node_modules/react-router-dom/esm/react-router-dom.js:1:17: error: No matching export in "node_modules/react-router/es/index.js" for import "__RouterContext"
node_modules/react-router-dom/esm/react-router-dom.js:1:34: error: No matching export in "node_modules/react-router/es/index.js" for import "matchPath"
node_modules/react-router-dom/esm/react-router-dom.js:2:9: error: No matching export in "node_modules/react-router/es/index.js" for import "MemoryRouter"
node_modules/react-router-dom/esm/react-router-dom.js:2:23: error: No matching export in "node_modules/react-router/es/index.js" for import "Prompt"
node_modules/react-router-dom/esm/react-router-dom.js:2:56: error: No matching export in "node_modules/react-router/es/index.js" for import "StaticRouter"

error goes away if upgraded to react-router ^5.2.0

aubreyarcangel commented 3 years ago

@JoeTice @pjhill I was not able to successfully replace webpack with esbuild.

As it stands, I have 132 out of 144 specs run successfully. Here's the status of each one. Here's my WIP branch for reference Bellow is the the status of each spec.

To get this running,

Step 1.

yarn add path-browserify yarn add -D esbuild @bahmutov/cypress-esbuild-preprocessor

Step 2. View my WIP branch and make all file changes

Step 3. This is the only change that wasn't reflected. Create a jsconfig.json in the root directory and drop in the json below:

{
    "compilerOptions": {
        "baseUrl": ".",
        "paths": {
            "*": [
                "*",
                "src/*",
                "src/platform/*",
                "src/applications/*",
                "src/site/*",
                "node_modules/*",
            ],
            "~/*": [
                "./src/*",
                "./src/platform/*",
                "src/*",
                "node_modules/*"
            ],
            "@@profile/*": [
                "./src/applications/personalization/profile/*"
            ],
            "@@vap-svc/*": [
                "./src/platform/user/profile/vap-svc/*"
            ],
            "@": [
                "node_modules/*"
            ]
        }
    }
}

Step 4. Load content-build and vets-website build

Step 5. run DEBUG=cypress-esbuild-preprocessor yarn cy:open

Cypress should now be using esbuild the preprocessor when specs are ran.

Bellow is the the status of each spec.

|Test                                                                                                                                |Error                                                                    |
|------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------|
|src/applications/appeals/10182/tests/10182-notice-of-disagreement.cypress.spec.js                                                   |passes                                                                   |
|src/applications/ask-a-question/tests/cypress/ask-a-question.authed.cypress.spec.js                                                 |does not run                                                             |
|src/applications/ask-a-question/tests/cypress/ask-a-question.unauthed.cypress.spec.js                                               |does not run                                                             |
|src/applications/caregivers/tests/e2e/1010cg.cypress.spec.js                                                                        |passes                                                                   |
|src/applications/coronavirus-research/tests/reviewAndSubmit.cypress.spec.js                                                         |passes                                                                   |
|src/applications/coronavirus-vaccination-expansion/tests/coronavirus-vaccination-expansion.cypress.spec.js                          |passes                                                                   |
|src/applications/coronavirus-vaccination-expansion/tests/coronavirus-vaccination-expansion-form-validation.cypress.spec.js          |passes                                                                   |
|src/applications/coronavirus-vaccination-expansion/tests/coronavirus-vaccination-expansion.cypress.spec.js                          |passes                                                                   |
|src/applications/coronavirus-vaccination/tests/e2e/hideauth.cypress.spec.js                                                         |passes                                                                   |
|src/applications/coronavirus-vaccination/tests/e2e/signup.cypress.spec.js                                                           |passes                                                                   |
|src/applications/coronavirus-vaccination/tests/e2e/unsubscribe.cypress.spec.js                                                      |passes                                                                   |
|src/applications/coronavirus-vaccination/tests/e2e/unsubscribeFail.cypress.spec.js                                                  |passes                                                                   |
|src/applications/debt-letters/tests/e2e/debt-letters.cypress.spec.js                                                                |does not run                                                             |
|src/applications/debt-letters/tests/e2e/diary-codes-content.cypress.spec.js                                                         |does not run                                                             |
|src/applications/debt-letters/tests/e2e/diary-codes-next-step.cypress.spec.js                                                       |does not run                                                             |
|src/applications/disability-benefits/2346/tests/mdot.cypress.spec.js                                                                |passes                                                                   |
|src/applications/disability-benefits/686c-674/tests/e2e/686C-674-ancilliary.cypress.spec.js                                         |passes                                                                   |
|src/applications/disability-benefits/686c-674/tests/e2e/686C-674.cypress.spec.js                                                    |passes                                                                   |
|src/applications/disability-benefits/996/tests/hlr-wizard.cypress.spec.js                                                           |passes                                                                   |
|src/applications/disability-benefits/996/tests/hlr.cypress.spec.js                                                                  |  > global is not defined                                                |
|src/applications/disability-benefits/all-claims/tests/all-claims-wizard.cypress.spec.js                                             |passes                                                                   |
|src/applications/disability-benefits/all-claims/tests/all-claims.cypress.spec.js                                                    |passes                                                                   |
|src/applications/disability-benefits/view-payments/tests/e2e/view-payments.cypress.spec.js                                          |passes                                                                   |
|src/applications/discharge-wizard/tests/e2e/discharge-wizard.cypress.spec.js                                                        |passes                                                                   |
|src/applications/edu-benefits/0994/tests/e2e/edu-0994.cypress.spec.js                                                               |> global is not defined                                                  |
|src/applications/edu-benefits/10203/tests/edu-10203.cypress.spec.js                                                                 |passes                                                                   |
|src/applications/edu-benefits/1990/tests/e2e/edu-1990.cypress.spec.js                                                               |passes                                                                   |
|src/applications/edu-benefits/1990e/tests/e2e/edu-1990e.cypress.spec.js                                                             |passes                                                                   |
|src/applications/edu-benefits/1990n/tests/e2e/edu-1990n.cypress.spec.js                                                             |passes                                                                   |
|src/applications/edu-benefits/1990s/tests/vrrap.cypress.spec.js                                                                     |> global is not defined                                                  |
|src/applications/edu-benefits/1995/tests/e2e/edu-1995.cypress.spec.js                                                               |passes                                                                   |
|src/applications/edu-benefits/5490/tests/e2e/edu-5490.cypress.spec.js                                                               |passes                                                                   |
|src/applications/edu-benefits/5495/tests/e2e/edu-5495.cypress.spec.js                                                               |passes                                                                   |
|src/applications/edu-benefits/tests/edu-apply-wizard.cypress.spec.js                                                                |> process is not defined                                                 |
|src/applications/facility-locator/tests/e2e/accessibility.cypress.spec.js                                                           |passes                                                                   |
|src/applications/facility-locator/tests/e2e/ccpDisabled.cypress.spec.js                                                             |passes                                                                   |
|src/applications/facility-locator/tests/e2e/ccpEnabled.cypress.spec.js                                                              |passes                                                                   |
|src/applications/facility-locator/tests/e2e/detailPage.cypress.spec.js                                                              |passes                                                                   |
|src/applications/facility-locator/tests/e2e/errorMessages.cypress.spec.js                                                           |passes                                                                   |
|src/applications/facility-locator/tests/e2e/facilitySearch.cypress.spec.js                                                          |passes                                                                   |
|src/applications/facility-locator/tests/e2e/gaEvents.cypress.spec.js                                                                |passes                                                                   |
|src/applications/facility-locator/tests/e2e/geolocation.cypress.spec.js                                                             |passes                                                                   |
|src/applications/facility-locator/tests/e2e/map-zoom.cypress.spec.js                                                                |passes                                                                   |
|src/applications/facility-locator/tests/e2e/mobile.cypress.spec.js                                                                  |fails on last test.                                                      |
|src/applications/facility-locator/tests/e2e/providerSearch.cypress.spec.js                                                          |passes                                                                   |
|src/applications/financial-status-report/tests/e2e/fsr-5655-wizard.cypress.spec.js                                                  |passes                                                                   |
|src/applications/financial-status-report/tests/e2e/fsr-5655.cypress.spec.js                                                         |does not run                                                             |
|src/applications/find-forms/tests/e2e/find-forms-required.cypress.spec.js                                                           |passes                                                                   |
|src/applications/gi/tests/e2e/00-required.cypress.spec.js                                                                           |react-router-dom & react-router error                                    |
|src/applications/gi/tests/e2e/02-profile-error.cypress.spec.js                                                                      |passes                                                                   |
|src/applications/gi/tests/e2e/03-vet-tec.cypress.spec.js                                                                            | react-router-dom & react-router error                                   |
|src/applications/gi/tests/e2e/04-dea.cypress.spec.js                                                                                |react-router-dom & react-router error                                    |
|src/applications/gi/tests/e2e/05-ojt.cypress.spec.js                                                                                |react-router-dom & react-router error                                    |
|src/applications/gi/tests/e2e/06-vet-tec-mobile.cypress.spec.js                                                                     |react-router-dom & react-router error                                    |
|src/applications/gi/tests/e2e/06-vet-tec-mobile.cypress.spec.js                                                                     |react-router-dom & react-router error                                    |
|src/applications/hca/tests/hca.cypress.spec.js                                                                                      |passes but doesn't finish                                                |
|src/applications/health-care-questionnaire/list/tests/e2e/print/failed.print.cypress.spec.js                                        |passes                                                                   |
|src/applications/health-care-questionnaire/list/tests/e2e/print/successful.print.cypress.spec.js                                    |passes                                                                   |
|src/applications/health-care-questionnaire/list/tests/e2e/00.home.cypress.spec.js                                                   |passes                                                                   |
|src/applications/health-care-questionnaire/list/tests/e2e/01.tabs.switch.cypress.spec.js                                            |passes                                                                   |
|src/applications/health-care-questionnaire/list/tests/e2e/02.bread.crumbs.cypress.spec.js                                           |does not run                                                             |
|src/applications/health-care-questionnaire/list/tests/e2e/03.is.downtime.notifier.cypress.spec.js                                   |passes                                                                   |
|src/applications/health-care-questionnaire/list/tests/e2e/03.session.storage.is.clearing.cypress.spec.js                            |passes                                                                   |
|src/applications/health-care-questionnaire/list/tests/e2e/03.session.storage.is.populated.cypress.spec.js                           |passes                                                                   |
|src/applications/health-care-questionnaire/list/tests/e2e/03.session.storage.is.populated.cypress.spec.js                           |passes                                                                   |
|src/applications/health-care-questionnaire/list/tests/e2e/04.questionnaire.happy.path.cypress.spec.js                               |does not run                                                             |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/bugs/questionnaire.unauth.loading.bug.cypress.spec.js            |passes                                                                   |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/bugs/reason.for.visit.bug.cypress.spec.js                        |passes                                                                   |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/bugs/unauthed.no-id.form.should.redirect.disabled.cypress.spec.js|passes                                                                   |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/bugs/unauthed.no-id.form.should.redirect.enabled.cypress.spec.js |passes                                                                   |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/content/health.care.questionniare.footer.loads.cypress.spec.js   |passes                                                                   |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/required-fields/reason.for.visit.is.required.cypress.spec.js     |passes                                                                   |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/00.questionnaire.is.disabled.cypress.spec.js                     |passes                                                                   |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/00.questionnaire.loads.cypress.spec.js                           |passes                                                                   |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/01.questionnaire.demographics.address.cypress.spec.js            |passes                                                                   |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/01.questionnaire.demographics.basicInformation.cypress.spec.js   |passes                                                                   |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/01.questionnaire.demographics.phoneNumbers.cypress.spec.js       |passes                                                                   |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/02.questionnaire.form.happy.path.cypress.spec.js                 |> global is not defined                                                  |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/03.questionnaire.landing.page.accessibility.cypress.spec.js      |passes                                                                   |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/04.appointment.id.is.missing.cypress.spec.js                     |passes                                                                   |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/04.appointment.id.is.present.cypress.spec.js                     |passes                                                                   |
|src/applications/health-care-questionnaire/questionnaire/tests/e2e/08.appointment.is.in.past.cypress.spec.js                        |passes                                                                   |
|src/applications/lgy/coe/tests/coe.cypress.spec.js                                                                                  |cy.route() cannot be invoked before starting the cy.server()             |
|src/applications/messages/tests/messages.cypress.spec.js                                                                            |passes                                                                   |
|src/applications/messages/tests/messages.cypress.spec.js                                                                            |passes                                                                   |
|src/applications/personalization/dashboard-2/tests/e2e/benefits-of-interest.cypress.spec.js                                         |passes                                                                   |
|src/applications/personalization/dashboard-2/tests/e2e/benefits-of-interest.cypress.spec.js                                         |passes                                                                   |
|src/applications/personalization/dashboard-2/tests/e2e/claims-and-appeals.cypress.spec.js                                           |passes                                                                   |
|src/applications/personalization/dashboard-2/tests/e2e/claims-and-appeals.cypress.spec.js                                           |passes                                                                   |
|src/applications/personalization/dashboard-2/tests/e2e/in-progress-forms.cypress.spec.js                                            |passes                                                                   |
|src/applications/personalization/dashboard-2/tests/e2e/loa1.cypress.spec.js                                                         |passes                                                                   |
|src/applications/personalization/dashboard-2/tests/e2e/loa3.cypress.spec.js                                                         |passes                                                                   |
|src/applications/personalization/dashboard-2/tests/e2e/messaging-error.cypress.spec.js                                              |passes                                                                   |
|src/applications/personalization/dashboard-2/tests/e2e/mpi-connection-error.cypress.spec.js                                         |passes                                                                   |
|src/applications/personalization/dashboard-2/tests/e2e/not-in-mpi-error.cypress.spec.js                                             |passes                                                                   |
|src/applications/personalization/dashboard/tests/e2e/dashboard.all-widgets.cypress.spec.js                                          |passes                                                                   |
|src/applications/personalization/dashboard/tests/e2e/dashboard.find-benefits-empty.cypress.spec.js                                  |passes                                                                   |
|src/applications/personalization/dashboard/tests/e2e/dashboard.find-benefits-selected.cypress.spec.js                               |passes                                                                   |
|src/applications/personalization/dashboard/tests/e2e/dashboard.health-care-widgets.cypress.spec.js                                  |passes                                                                   |
|src/applications/personalization/dashboard/tests/e2e/dashboard.minimal-widgets.cypress.spec.js                                      |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/address-validation/bad-unit.cypress.spec.js                                      |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/address-validation/confirm-address.cypress.spec.js                               |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/address-validation/confirm-state-diff.cypress.spec.js                            |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/address-validation/edit-after-validation.cypress.spec.js                         |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/address-validation/international-address.cypress.spec.js                         |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/address-validation/low-confidence.cypress.spec.js                                |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/address-validation/military-address.cypress.spec.js                              |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/address-validation/missing-unit.cypress.spec.js                                  |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/address-validation/no-changes-made.cypress.spec.js                               |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/address-validation/one-suggestion.cypress.spec.js                                |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/address-validation/two-suggestions.cypress.spec.js                               |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/address-validation/valid-address.cypress.spec.js                                 |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/direct-deposit/gating.cypress.spec.js                                            |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/direct-deposit/non-2fa.cypress.spec.js                                           |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/direct-deposit/update-flow.cypress.spec.js                                       |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/personal-and-contact-info/fields-check.cypress.spec.js                           |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/personal-and-contact-info/mailing-address.cypress.spec.js                        |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/personal-and-contact-info/modals.cypress.spec.js                                 |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/personal-and-contact-info/vap-error.cypress.spec.js                              |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/loa1.cypress.spec.js                                                             |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/nametag.cypress.spec.js                                                          |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/profile-a11y.cypress.spec.js                                                     |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/profile.connected-apps.cypress.spec.js                                           |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/profile.mpi-error.cypress.spec.js                                                |passes                                                                   |
|src/applications/personalization/profile/tests/e2e/profile.not-in-mpi-error.cypress.spec.js                                         |passes                                                                   |
|src/applications/personalization/rated-disabilities/tests/e2e/rated-disabilities.cypress.spec.js                                    |passes                                                                   |
|src/applications/personalization/view-dependents/tests/e2e/view-dependents.cypress.spec.js                                          |passes                                                                   |
|src/applications/personalization/view-representative/tests/view-representative.cypress.spec.js                                      |Runs, but displays error: No elements found for include in frame Context |
|src/applications/search/tests/e2e/00-required.cypress.spec.js                                                                       |passes                                                                   |
|src/applications/static-pages/dependency-verification/tests/dependency-verification.cypress.spec.js                                 |passes                                                                   |
|src/applications/static-pages/health-care-manage-benefits/schedule-view-va-appointments-page/tests/index.cypress.spec.js            |passes                                                                   |
|src/applications/vaos/tests/e2e/appointment-list.cypress.spec.js                                                                    |passes                                                                   |
|src/applications/vaos/tests/e2e/community-care.cypress.spec.js                                                                      |passes                                                                   |
|src/applications/vaos/tests/e2e/covid19-vaccine.cypress.spec.js                                                                     |passes                                                                   |
|src/applications/vaos/tests/e2e/va-appointment.cypress.spec.js                                                                      |passes                                                                   |
|src/applications/vaos/tests/e2e/va-request.cypress.spec.js                                                                          |passes                                                                   |
|src/applications/virtual-agent/tests/virtual-agent.cypress.spec.js                                                                  |passes                                                                   |
|src/applications/vre/28-1900/tests/e2e/chapter31-loa1.cypress.spec.js                                                               |passes                                                                   |
|src/applications/vre/28-1900/tests/e2e/chapter31-maximal.cypress.spec.js                                                            |passes                                                                   |
|src/applications/vre/28-1900/tests/e2e/chapter31-wizard.cypress.spec.js                                                             |passes                                                                   |
|src/applications/vre/28-8832/tests/e2e/dependent-workflow.cypress.spec.js                                                           |passes                                                                   |
|src/applications/vre/28-8832/tests/e2e/veteran-workflow-not-logged-in.cypress.spec.js                                               |passes                                                                   |
|src/applications/yellow-ribbon/e2e/tests/yellow-ribbon.cypress.spec.js                                                              |passes                                                                   |
|src/platform/site-wide/side-nav/tests/e2e/sideNav.cypress.spec.js                                                                   |passes                                                                   |
|src/platform/site-wide/user-nav/tests/e2e/00-required.cypress.spec.js                                                               |passes                                                                   |

The remaining errors and my guess on the issues: The handful of tests that don't run may have something to do with my local build. global is not defined - missing dependency. webpack uses core-js-pure to define global process is not defined missing dependency. various react-router and react-router-dom errors - outdated dependcy. errors clears when react-router and react-router-dom are updated to version 5

U-DON commented 3 years ago

Work done

Attempted to recreate the changes with some modifications here: department-of-veterans-affairs/vets-website/pull/17621

Main differences from previous WIP branch:
- Leveraged more built-in options or third-party plugins to properly resolve modules.
- Some refactoring and cleanup of lint issues and unused code.
Note that this branch was also based on top of another branch that uses esbuild via a Webpack loader for the full build, but that was irrelevant to the preprocessing changes.

Test failures

When run locally, 13 of 155 tests failed.

Facility Locator tests failed to import stylesheets that were included in the bundle.
- applications/facility-locator/tests/e2e/accessibility.cypress.spec.js
- applications/facility-locator/tests/e2e/ccpDisabled.cypress.spec.js
- applications/facility-locator/tests/e2e/ccpEnabled.cypress.spec.js
- applications/facility-locator/tests/e2e/mobile.cypress.spec.js
GIBCT tests failed due to incorrect react-router import resolution.
- applications/gi/tests/e2e/00-required.cypress.spec.js
- applications/gi/tests/e2e/03-vet-tec.cypress.spec.js
- applications/gi/tests/e2e/04-dea.cypress.spec.js
- applications/gi/tests/e2e/05-ojt.cypress.spec.js
- applications/gi/tests/e2e/06-vet-tec-mobile.cypress.spec.js
- One test failing due to selector not being found. Root issue unknown.
- applications/appeals/10182/tests/10182-notice-of-disagreement.cypress.spec.js
Remaining failing tests were incomplete or placeholder tests meant to be skipped in CI.
- These tests are skipping based on the condition Cypress.env('CI').

Observations and findings

react-router seems to be incorrectly getting resolved to the one from the top level node_modules.
- vets-website uses react-router@3, but react-router-dom uses react-router@^5.2.0.
- It should be resolving to the closest node_modules to the modules (in react-router-dom) that are importing it.
- Upgrading is not the recommended approach to fixing this.
  - It's a matter of resolving to the right version of the package. Webpack handles the resolution without issue, so ideally we'd use esbuild's features to accomplish the same thing.
  - Going from v3.x.x to v5.2.0 involves jumping two major versions. There were major breaking changes in v4, and we'd have to migrate how the router is used across all apps.
Some facility-locator tests were failing on importing stylesheets (.scss).
- Attempted to use esbuild-plugin-sass, esbuild-sass-plugin, etc. without success.
- It seems that importing from node_modules past the first level seems to fail.
- Past the first level, the resolver starts searching paths from the project root for some reason.
```
> src/platform/site-wide/user-nav/index.js:11:7: error: [plugin: sass-plugin] Cannot find module 'uswds/src/stylesheets/lib/bourbon' from '/Users/eugenedoan/Code/vets-website'
```
Import paths like platform/a/b/c or applications/a/b/c seem to resolve fine locally but failed in CI.
- I set the nodePaths in the config, but it doesn't seem to work in CI.
- Maybe could set the NODE_PATH environment variable?

Conclusion

Dropping this work for now given the number of issues and the amount of effort required to fix them all.

esbuild and its ecosystem is also still evolving and hasn't yet reached v1.0.0.

pjhill commented 3 years ago

Thanks for the excellent investigation and write-up @U-DON! We will revisit this potential swap in Q4 here.

department-of-veterans-affairs / va.gov-team

Research swapping Webpack out for ESBuild #23565

Description

Tasks

Acceptance Criteria

Work done

Test failures

Observations and findings

Conclusion