NeuroBench / neurobench

Benchmark harness and baseline results for the NeuroBench algorithm track.
https://neurobench.readthedocs.io
Apache License 2.0
52 stars 12 forks source link

Metric output multiple values #113

Closed jasonlyik closed 11 months ago

jasonlyik commented 11 months ago

@korneelf1 The synaptic ops metric would benefit from discerning between MACs (because of float activations) vs ACs (because of binary activation). This could be implemented in the synops metric by checking whether the layer output is binary.

We could make the synops an AccumulatedMetric which saves the state of both MACs and ACs and returns a dict as output, so metric out would be 'synaptic_operations': {'MACs': X, 'ACs': Y}.

For the baselines, this would only be relevant for the Hybrid ANN/SNN for the obj detection, as all other models are doing either all MACs or all ACs.

korneelf1 commented 11 months ago

Done in effective_macs branch

Remarks: