search

microsoft / knossos-ksc

Compiler with automatic differentiation
Other
45 stars 10 forks source link

Don't emit tensor values in benchmark JSON #1028

Open awf opened 3 years ago

awf commented 3 years ago

[Edited - I wrongly worried about file size]

Currently the pytest-benchmark JSON outputs are emitting the tensor values (with ... for large tensors) on which benchmarks are run.
image

Proposal: don't emit the config parameters

Pro: slightly smaller benchmark JSON files.

Con: Can't check that two benchmarks ran on same values?

awf commented 3 years ago

@cgravill, thoughts?

cgravill commented 3 years ago

Yes, partial values don't help much anyway so skipping might be easier. I'd have to investigate to see how to avoid outputting though.

Outputting the seed might help with reproducibility (if we were to properly pin versions etc.) but is separate.