TrueCloudLab / xk6-frostfs

k6 extension to test and benchmark NeoFS related protocols.
GNU General Public License v3.0
20 stars 4 forks source link

Add local targets #14

Open fyrchik opened 1 year ago

fyrchik commented 1 year ago

Sometimes we want to perform benchmark on a filled cluster. Filling it with gRPC is fast, but not enough: we are talking about weeks, not hours. Writing a separate utility for this is a dead-end scenario IMO: it has only 1 use-case, maintenance costs are non-negligible, users need to learn yet another CLI interface (and, of course, inevitable bugs in the implementation).

The proposal is to use implement local client:

  1. Use engine from frostfs-node.
  2. Execute k6 on a target hardware using new local.js scenario.
  3. Policer/tree service sync will do the rest.

Pros:

  1. The results can be compared with gRPC results to see the overhead we have.
  2. Easily automated to test for regressions. Local benchmarks in the frostfs-node are not suitable for this: they are usually not long/big enough.
  3. S3 is also easily supported: engine exports all necessary machinery.

Cons:

  1. Harder to maintain compatibility: we need to update k6 each time engine is updated. However, it isn't impossible: the engine interface changes very rarely.
  2. Importing frostfs-node here is not aestethically pleasing IMO.

cc @anikeev-yadro @realloc @carpawell @alexvanin

realloc commented 1 year ago

S3 is also easily supported: engine exports all necessary machinery.

Could you please clarify here a bit? Do you mean handling both tree service calls and storage engine calls in "S3-local" case?

fyrchik commented 1 year ago

Yes. Tree service is just a wrapper over the engine, we can work with pilorama directly. The only difficulty is knowing the tree structure for a bucket, but our bench is simple enough (flat structure, single version).

realloc commented 1 year ago

There is one more benchmark for versioned objects case where we put a loooong chain of versions for an object. Ok. LGTM =)