auto testing by Github

[jbellik]

Goal: To take advantage of Github's "Actions" to automatically run various tests whenever a pull request comes in.

To do this, we need all the (relevant) Mocha tests to be in .js files by themselves.

• I think the tests can then be called from the existing html test files to provide a visual / more detailed information for when they fail.
• Each JS test file with Mocha tests in it can be compiled into a big test.js file (akin to constraints.js).
• Then the workflow yml script will run test.js every time a pull request comes in.

[jbellik]

Sub tasks:

• [x] pull Mocha tests out of existing html auto test files into separate .js files
• [x] change existing auto test html files to run the code in the .js files
• [x] write a new testbuild.sh script, or modify the existing one
• [x] change the yml script to use test.js

[jbellik]

First step would be to try pulling the Mocha code/script out of one of the existing automated test files and adding a line to the workflow that runs it

[sz245]

When pulling mocha tests out of existing html into .js files, do you mean find a command that does that or change the test files manually like with the interface test files or either or?

[jbellik]

As a first step, let's take the mocha tests out manually from one html file. That will give us a .js file, let's call it test1.js, that (I hope) can be used both in the html file and in the Github action (in the yml file). Then try adding a line to the .yml file that runs test1.js.

[sz245]

start with stringTest.js

[sz245]

branch github

[jbellik]

Since stringTest.js is using the interface, let's actually start with extracting the script from binarityTest.html into binarityTest.js.

[jbellik]

Possibly we also need to install chai in the yml script.

[jbellik]

mocha.setup("bdd"); //brings "describe", "it", etc. into global namespace //mocha.checkLeaks(); var assert = chai.assert;

Ultimately these lines also need to get into the yml or into the compiled test.js file, but for now, just grab them along with the rest of the binarity tests and put them at the top of binarityTest.js.

[sz245]

It seems that I need to first merge my code with master branch to be able to test it in github actions. Is there another way to test it first without merging?

[sz245]

https://gist.github.com/soheilhy/867f76feea7cab4f8a84 https://www.freecodecamp.org/news/what-are-github-actions-and-how-can-you-automate-tests-and-slack-notifications/ https://docs.github.com/en/actions https://github.com/actions/checkout/blob/main/.github/workflows/test.yml https://stackoverflow.com/questions/9722407/how-do-you-install-and-run-mocha-the-node-js-testing-module-getting-mocha-co https://itnext.io/getting-started-with-github-actions-fe94167dbc6d

[jbellik]

Some general info on using Mocha in the terminal:

• If a package.json file is present, you can specify test: "mocha". Then running npm test will automatically run all the js files in the test directory: test/*.js
• Since we haven't set SPOT up to be modularized, we have to put spot.js in test.js in order to access all the functions without messing around with refactoring to use import or require.

[jbellik]

Largely addressed by #567 Running mocha in the console requires installing node. If you have node installed on your local machine, and have pulled package.json into your local repository, you can check whether all the tests in main/test/compiledTest/test.js are passing: Step 1: From main, run ./mochabuild.sh. (NB I have not written a .bat version of this build script!) This will create test.js in the directory main/test/compiledTest. Step 2: Also from main, run npm test, which will evaluate the 400+ tests in test.js and output results to the command line. But Github will also run all those tests for you when you set up a pull request to master, if you don't want to set node up locally. I did add node_modules to the .gitignore file so they don't end up in the repository.

Remaining tasks to finish setting up Github's auto testing:

• [x] extract test functionality from the following files and put it in corresponding JS files, wrapped in a function that the html file calls:

• • [x] binMaxHeadTest.html -- that one is related to the headmarking debugging that Ed is working on @3ddiehead
• • [x] narityTest.html @M130
• • [x] noShiftGradientTest.html
• • [x] strongStartHsuTest.html
• • [x] syntacticFiltersTest.html. @M130

• See other test files for working examples. Ex: binarityTest.js and binarityTest.html

• [x] Add describe() statements around the entire set of tests in each test function, naming the test file that the code comes from

• [x] update jsbuild.bat to match jsbuild.sh. It should make top.js as a distinct build file, consisting of all the .js files in the top level of main. This is because top.js needs to go into compiledTest/test.js for mocha testing, but we don't want interface.js to go in it because it makes reference all over the place to the window/browser which the Github auto testing cannot handle. @3ddiehead

• [x] update testBuild.bat to match testBuild.sh: make a build/test.js that excludes the mocha files, since we don't want those in compiledTest/test.js either, since we install mocha separately in that virtual machine @3ddiehead

• [x] write .bat version of mochabuild.sh -- mochabuild.bat @3ddiehead

[jbellik]

@M130 Could you work on this some more? Take a look at the post just above this and let me know if you have questions. Basically you can test it all locally if you install node; I set up the package.json file and changed the .gitignore to ignore the node files, so it won't mess up the repository even if you put it in your local repository folder.

I'm thinking it should be pretty easy to deal with the rest of the test files (listed in the post above) and you could probably manage that before Wednesday. But you'll need to update the .bat files to test it, if I'm remembering correctly that you're on a Windows machine.