The Web Latency Benchmark is a new kind of benchmark that tests your browser's responsiveness by directly measuring latency and jank. Visit the homepage at http://google.github.io/latency-benchmark for examples of the kinds of latency and jank that are measured.
The Oculus Latency Tester is a hardware device with a light sensor that can measure end-to-end latency from USB input to pixels changing on the screen. This kind of hardware-based measurement accounts for all possible sources of latency. It's the most complete and accurate measurement possible, and it's now supported by the Web Latency Benchmark. Just plug it in and you'll see a special test page.
Thanks to jmaher, the benchmark now accepts command-line arguments that enable fully automated benchmark runs, with results reported in JSON format to a server of your choosing.
The Web Latency Benchmark works by programmatically sending input events to a browser window, and using screenshot APIs to detect when the browser has finished drawing its response.
There are two main components: the latency-benchmark server (written in C/C++) and the HTML/JavaScript benchmark page. The HTML page draws a special pattern of pixels to the screen using WebGL or Canvas 2D, then makes an XMLHTTPRequest to the server to start the latency measurement. The server locates the browser window by searching a screenshot for the special pattern. Once the browser window is located, the server starts sending input events. Each time the HTML page receives an input event it encodes that information into pixels in the on-screen pattern, drawn using the canvas element. Meanwhile the server is taking screenshots every few milliseconds. By decoding the pixel pattern the server can determine to within a few milliseconds how long it takes the browser to respond to each input event.
The native reference test is special because it requires extra support from the server. Using the native APIs of each platform, the server creates a special benchmark window that draws the same pattern as the test webpage, and responds to keyboard input in the same way. To ensure fairness when compared with the browser, this window is opened in a separate process and uses OpenGL to draw the pattern on the screen. The benchmark window opens as a popup window, only 1 pixel tall and without a border or title bar, so it's almost unnoticeable.
The Web Latency Benchmark is licensed under the Apache License version 2.0. This is an open source project; it is not an official Google product.
Python 2.x is required on all platforms for GYP, which generates the build files.
First, you need to git submodule init && git submodule update
to fetch the submodules in third_party. Then, you need to run generate-project-files
, which will run GYP and generate platform-specific project files in build/.
build/latency-benchmark.sln
.build/latency-benchmark.xcodeproj
. For debugging you will need to edit the default scheme to change the working directory of the latency-test
executable to $(PROJECT_DIR)
so it can find the HTML files. You will also want to configure the debugger to ignore SIGPIPE.linux-build
to compile with Clang. The binary will be built at build/out/Debug/latency-benchmark
. Run it in the top-level directory so it can find the HTML files. You can build the release version by defining the environment variable BUILDTYPE=Release
.You shouldn't make any changes to the XCode or Visual Studio project files directly. Instead, you should edit latency-benchmark.gyp
to reflect the changes you want, and re-run the generate-project-files
script to update the project files with the changes. This ensures that the project files stay in sync across platforms.