thekevinscott
commented
9 months ago the live site is down
What do you mean by this?
Some context on why I removed these scripts.
Speed measurements are highly device-specific. The speed tests were leveraging Browserstack to perform inference on mobile devices, so there was at least some consistency in hardware, but there's huge variability in terms of what platforms are available, what version of software, even what's running at the same time. That combined with the dependency on Browserstack (which is also used for integration testing, and where we often ran out of slots) made me want to remove that.
Regarding performance measurements, I'm increasingly of the mind to move away from library-specific training of models and embracing pre-trained models converted into Javascript. (See #842.) In that case, it makes sense to refer to the original model repository or paper's references on performance, as they should be equivalent on a relative level.
Finally, I'd not run this particular benchmarking code in over a year and it would require some refactoring to bring up to date with the rest of the repository. I was not under the impression anyone was using it.
With that all said, I'd love to hear your perspective. Is benchmarking important to you? Are you relying on it in some way?
wes-kay
With benchmarking is gone and the live site is down, is there anywhere to get any info on benchmarking without having to manually, do it?
Thanks,