testRunner compiles incorrectly... *sometimes*

[spencercorwin]

Description

Webpack's compiling of the testRunner for the website will sometimes produce something that doesn't work

Steps to Reproduce

1. Compile the website with yarn website:compile:all:local
2. In separate terminal tabs run: yarn website:deploy-static:local, yarn website:deploy-preview:local, yarn website:deploy-server:local, yarn website:deploy-testRunner:local
3. Navigate to localhost:3000, go to the Courses, go to Lesson 1 Chapter 2, enter the solution, build, then run the tests. The tests may or may not fail with this message: token.symbol is not a function.
4. If the tests pass then the testRunner likely works and you need to recompile the testRunner by shutting it down and running rm -rf ./dist/testRunner, then yarn website:start-testRunner-prod or yarn run ts-node ./scripts/website/compile --bundle testRunner && yarn website:deploy-testRunner:local. Reset the browser and check the testRunner again until you see the error.

Expected behavior: testRunner works every time it's compiled

Actual behavior: testRunner works sometimes

Reproduces how often: ~30%

Additional Information

I was unable to produce a failing version of the testRunner when passing 'dev' into const compileTestRunner = async () => compileConfig(testRunner({ stage: 'prod' })); on line 104 of compile.ts. This may provide a lead as to a webpack configuration that could be causing this.

I tried turning webpack's minimizer off, then producing a failing testRunner dist and a succeeding testRunner dist, then comparing some of the files between the two. This didn't reveal any insights.

This is what the failing test looks like:

[spencercorwin]

Possibly due to webpack optimizations? https://github.com/webpack/webpack/issues/8435

[spencercorwin]

Definitely related to this