search

MystenLabs / walrus-sites

Walrus Sites: Decentralized Websites using Sui and Walrus.
https://docs.walrus.site/walrus-sites/intro.html
Apache License 2.0
19 stars 16 forks source link

Define performance metrics - benchmarks #185

Closed Tzal3x closed 1 week ago

Tzal3x commented 2 weeks ago

Description

Define and calculate the performance metrics for our portals.

We need to measure how fast a site is served so that we can use it as a baseline, and improve on it as time goes on.

It's important to distinguish metrics which are affected by how a website is implemented and the ones affected by the portal logic (site::Resource resolution etc).

Component

portal

Code of Conduct

Tzal3x commented 2 weeks ago

The part of which we should profile is the portal fetching the site::Resources from the aggregator.

For starters, we could just monitor the average response times of the server portal given a constant walrus.site like the snake example, or the landing page.

A tool to do this is k6, but I've asked around Mysten if there are other performance monitoring tools that are preferred.

Tzal3x commented 2 weeks ago

Asked around production engineering for recommendations. Quoting:

Looks like vercel uses otel collector to send traces. its in early stages for the vendor we use (Datadog). Its also unclear if we can get the exporter to send traces to Tempo. https://vercel.com/docs/observability/otel-overview

Tzal3x commented 2 weeks ago

There are two ways to do performance testing for our portals:

  1. Macro-testing: compare the responses time of a walrus site served by the server-portal (running on vercel) and the same site served directly by vercel (without the involvement of walrus). The "health check" site could be part of this.
  2. Micro-testing: benchmarking specific functions such as fetchPage, fetchResource. A library to do this in typescript would be this one: https://github.com/microsoft/typescript-benchmarking. The benchmarking tests could also be included in CI/CD in order to monitor if changes we do affect the performance of the portal.

We decided to start with micro-testing, as we already know the time-expensive functions and then proceed with the macro.

Tzal3x commented 2 weeks ago

Vitest also supports benchmarking: https://vitest.dev/guide/features#benchmarking.

Since we already use vitest for our tests, it would be worth checking out.