search

CodSpeedHQ / codspeed-node

Node.js libraries to create CodSpeed benchmarks
https://codspeed.io
Apache License 2.0
12 stars 2 forks source link

Profiling doesn't work with tinybench plugin #37

Open XantreDev opened 1 month ago

XantreDev commented 1 month ago

Profiling doesn't work with tinybench image profiling source

Profiling works with vitest image profiling source

adriencaccia commented 2 weeks ago

Hey @XantreDev, thanks for the issue.

I managed to dig a bit and it seems that the problem is a mix of reasons: using yarn, the CodSpeed action and tinybench with TS.

Investigation

We can see here that the node introspection is not working, the node flags are set only on the yarn bench command and not the underlying node ... command.

When manually calling the node ... command instead of yarn bench in the GH workflow file, the introspection works as expected (see here).

We did not face this problem before because most users are using pnpm, which actually runs the underlying node ... command when executing a script.

Workaround

The process is much simpler when using vitest and it works as expected, even with yarn.

I will keep this issue open, since we might want a fix in the future if others face the same issue again.