Puppeteer tests for Odyssey

[elmisback]

Tests to write for a test suite:

• [ ] Test everything in Ben’s presentation

  • [ ] Reach out to Ben

What features are expected? Everything should appear Possible variation in accuracy How do we see if an expression works? Check if points look like what we expect What’s the definition here? Use a good proxy: Accuracy roughly the same Speedup roughly the same Loose test: using image processing language model to verify if previous image VS current image are similar in nature

• [ ] Get a utility function for generic test (test harness with accuracy and speedup metrics)

Test arbitrary functions Maybe set up a config file for arbitrary functions

• [ ] Concrete values testing

• [ ] Hover over local error to get stats on this stuff

• [ ] Stress testing

Send in a bunch of requests

[NoxNovus]

Meeting with Ben on Friday to discuss this. Otherwise, have basic tests working with component fetching for accuracy and speedup, and currently working on refactoring it into a more generalizable test suite.

[NoxNovus]

Met with Ben, I have expressions, next step is to get specific demo workflows from Ben (probably will contain local error, accuracy, speedup, and improve button) and write specific tests for this - have a set of tests for basic expressions ready.

[NoxNovus]

A specific demo workflow including Herbie improve button functionality is ready. Also, merged main into automated-testing branch, and automated-testing back into main.

[NoxNovus]

Next is testing for concrete values, and testing for hover values. Puppeteer doesn't seem to have great support for the clickable circles in the errorplot, currently trying to figure out how to get it to click those properly.