Open rachel-fenichel opened 5 years ago
Working on this in PR #4903. Looks like the generator tests are fine on macOS, but mocha tests are failing in CI (not locally).
I looked at this a bit but I am unable to figure out what is going wrong. Comparing test logs from my own machine vs. our CI servers it looks like the CI servers are actually running MORE mocha tests than get run locally. Here are the test output files, trimmed to start with the mocha tests and modified to remove timestamps and element IDs for easier diffing:
npm run test:run
on my machine.Could it be that tests are passing locally only because we are not running all of them?
I'm going to leave this for the time being because I do not understand mocha, selenium, webdriver, etc.
One further possible clue about what is going on: it seems that the mocha tests are flaky on macOS, and sometimes succeed and sometimes not; e.g., for the same commit:
I don't think its v16 specific, however, because at earlier commits v12 passed but v14 and v16 failed.
To work on this bug:
macos-latest
back in build.yml
@BeksOmega looks like all tests passes on macos for now: https://github.com/alicialics/blockly/actions/runs/6830615442/job/18578823354?pr=2
Reverting #7640 in #7692 because CI does not pass consistently, but is unrelated to macos actually failing.
Generator tests require firefox instead of chrome, but our current setup has issues with geckodriver on osx. Low priority because we do regularly build on osx, and because ubuntu tests are running.