Open JCQuintas opened 1 month ago
Hey @JCQuintas, thanks for the issue.
It seems that there will be a bit of work to support browser testing within CodSpeed, too bad that the error is not really helpful 😕. I will need to investigate.
In the meantime, to still be able to leverage CodSpeed and detect performance changes, you can still set up CodSpeed using jsdom
for tests where it is possible. I expect that performance changes will still be picked up in this simulated environment, even if the relative values will be different.
Use Case
We want to benchmark the test in a real-browser environment to test closer to what the user will actually experience. Also, some of our planned tests would not work at all in a
jsdom
environment.We looked at codspeed to help us detect performance changes between pull request, as to not slow down the MUI X Charts library unexpectedly.
Issue
Tests break when running them with
codspeed-runner
using vitest in browser mode with playwrightThey break in the
CodSpeedHQ/action
, and when installing and runningcodspeed-runner
manually. But they don't break if we runvitest bench
reproducible repo: https://github.com/JCQuintas/codspeed-mui-example failed actions: https://github.com/JCQuintas/codspeed-mui-example/actions/runs/10074486926