search

DeepMark / deepmark

THE Deep Learning Benchmarks
Apache License 2.0
351 stars 41 forks source link

Torch DeepSpeech2 Benchmark multi-GPU #7

Closed SeanNaren closed 7 years ago

SeanNaren commented 7 years ago

Hey I've added multi-GPU support to the torch benchmark, hopefully it's useful. Let me know of any issues/feedback on the PR!

EDIT: should've also added I think we need some form of progress monitoring, because running the benchmark is a lengthy process (through the entire dataset). xlua alright for this?

SeanNaren commented 7 years ago

Thought I could sneak in a progress bar to the PR! @soumith currently it gives the same stats as the covnet benchmarks, do we want anything else (total time taken or something)?

soumith commented 7 years ago

ok this is cool. just need a stats reporting like: epoch time, total number of samples processed. samples / second. And some metric that reports a (normalized sample length) / second

SeanNaren commented 7 years ago

@soumith what do you mean (normalized sample length) / second?

soumith commented 7 years ago

The data samples are of different utterance lengths each. We want a metric that gives the throughput. So, [(seconds of audio processed) / second] for example would be a normalized metric

SeanNaren commented 7 years ago

Hopefully the added measurements are correct, here is a table including all the current headers

Hardware Time (ms) forward (ms) backward (ms) Samples processed Samples processed per second Seconds of audio processed per second Epoch time (s)
1x Titans - - - - - - -
4x Titans - - - - - - -

Does this cover everything needed? Let me know if anything needs changing etc.

EDIT: 5 runs enough for the average?

soumith commented 7 years ago

Thanks a ton Sean!