WIP to show how workloads from another repository could get installed and used in Speedometer, with minimal changes.
What is speedometer-workloads?
Speedometer-workloads is a standalone monorepo that contains example apps and utilities that can be installed and used, instead of the current workloads that are included in the speedometer benchmark itself.
Benefits of using speedometer-workloads
Decoupling workloads from the benchmark would allow more flexibility and control over what workloads to use for local testing and how they can be configured for these tests.
Having a more modern and dynamic development setup will promote speed of development and motivate experimentation of new workloads, without being forced to adhere to technical constraints of Speedometer itself.
Speedometer-workloads can adhere to its own release cycle and will establish a new versioning of these workloads, which will result in more granular control on what workloads to install for each release of Speedometer.
A nice side effect of this change is that the workloads would also be re-usable for other benchmarks, if the need arises in the future.
Workloads install
Workloads are now installed through a (private) npm package. Currently, one single package is used for all workloads, instead of installing each workload separately. The Benefit of using a single package is simplicity. Downside is that we can’t decide in Speedometer, which version of a specific workload should be installed. This should be an easy step to change though.
speedometer-workloads
This package should be installed as a regular dependency of Speedometer.
For testing purposes, the current package is private, until the changes are approved and agreed on.
Since the package is private, the npm package name is scoped: @tkober/speedometer-workloads.
Link to the speedometer-workloads source: https://github.com/GoogleChromeLabs/speedometer-workloads
The copy-workloads script in the package.json file, creates a workloads folder in the root of Speedometer with a copy of the dist folders of each work that is listed in the workloads.config.folder.json file.
"scripts": {
"copy-workloads": "npm explore @tkober/speedometer-workloads pnpm run move:apps:custom"
}
Benchmark Runner
_prepareSuite
Multiple promises have to be resolved for this step:
loadFrame: resolves once the iframe is done loading
suite.prepare: any additional preparation steps a workload might need
postMessageSent({ type: “app-ready”}): a message sent from the workload when it can accept a test
Compared to the previous implementation, we are now only appending an iframe and creating a Page class instance, if a suite is not disabled.
for (const suite of suites) {
if (!suite.disabled){
await this._appendFrame();
this._page = new Page(this._frame);
await this._runSuite(suite);
this._removeFrame();
}
}
Tests
Suites.push has been updated for each workload.
The url points to the workloads folder in the root of the repository.
Prepare function is now mostly just a stub, since the workloads send an “app-ready” message
WIP to show how workloads from another repository could get installed and used in Speedometer, with minimal changes.
What is speedometer-workloads?
Speedometer-workloads is a standalone monorepo that contains example apps and utilities that can be installed and used, instead of the current workloads that are included in the speedometer benchmark itself.
Benefits of using speedometer-workloads
Decoupling workloads from the benchmark would allow more flexibility and control over what workloads to use for local testing and how they can be configured for these tests.
Having a more modern and dynamic development setup will promote speed of development and motivate experimentation of new workloads, without being forced to adhere to technical constraints of Speedometer itself.
Speedometer-workloads can adhere to its own release cycle and will establish a new versioning of these workloads, which will result in more granular control on what workloads to install for each release of Speedometer.
A nice side effect of this change is that the workloads would also be re-usable for other benchmarks, if the need arises in the future.
Workloads install
Workloads are now installed through a (private) npm package. Currently, one single package is used for all workloads, instead of installing each workload separately. The Benefit of using a single package is simplicity. Downside is that we can’t decide in Speedometer, which version of a specific workload should be installed. This should be an easy step to change though.
speedometer-workloads
This package should be installed as a regular dependency of Speedometer. For testing purposes, the current package is private, until the changes are approved and agreed on. Since the package is private, the npm package name is scoped: @tkober/speedometer-workloads. Link to the speedometer-workloads source: https://github.com/GoogleChromeLabs/speedometer-workloads
workloads.config.folder.json
This file determines which workload should get copied over from the @tkober/speedometer-workloads package.
copy-workloads
The copy-workloads script in the package.json file, creates a workloads folder in the root of Speedometer with a copy of the dist folders of each work that is listed in the workloads.config.folder.json file.
Benchmark Runner
_prepareSuite
Multiple promises have to be resolved for this step:
_runAllSuites
Compared to the previous implementation, we are now only appending an iframe and creating a Page class instance, if a suite is not disabled.
Tests
Suites.push has been updated for each workload.
Example
Notes
@kara