search

NeuroBench / neurobench

Benchmark harness and baseline results for the NeuroBench algorithm track.
https://neurobench.readthedocs.io
Apache License 2.0
55 stars 12 forks source link

SynOp per model forward #137

Closed jasonlyik closed 11 months ago

jasonlyik commented 12 months ago

In the paper we will be defining SynOps as operations per model forward, rather than operations per sample. Therefore the synop metric should be divided over the 0 and 1 dimensions (currently it is only divided over the 0 dimension).

For now can do the division by hand, so not urgent.

jasonlyik commented 11 months ago

Just kidding, we are doing operations per sample so that it's not confusing in result report