search

mozilla-services / autograph

Mozilla's digital signature service
https://hub.docker.com/r/mozilla/autograph/
Mozilla Public License 2.0
151 stars 35 forks source link

load test and track Fx try,ngt, and rel workloads #347

Open g-k opened 5 years ago

g-k commented 5 years ago

run perf tests in CI to let us know if we introduce perf regressions in CI output (ideally like https://arewefastyet.com/)

g-k commented 5 years ago

For omni.ja signing Callek says:

the easiest way for a full one is to grab a firefox installer (windows .zip or linux .tar.bz2) and extract and find . -name omni.ja there will be two in there

as for the stripped down copy there is two types of omni.ja files in here -- https://github.com/mozilla-releng/signingscript/tree/master/tests/data preload and non-preload)

g-k commented 4 years ago

This could consist of:

  1. a tool to create dummy autograph configs with a certain number of signers and keys of given types (or migrate them to the DB and populate the DB accordingly)
  2. autograph client updates or another load testing tool that simulates load against a given set of signers (e.g. 10k reqs to signers X and Y while a monitor runs)
    1. a tool to generate these from stage and prod logs (e.g. replay high load scenario from two weeks agao)
    2. this could simulate poor client network conditions or malicious clients (refs: #382)
  3. profiling and monitoring in the server container at the OS and golang runtime layers (refs: #384 and #385 respectively)
  4. running the tests in CI, saving the results, and failing on major perf regressions
g-k commented 4 years ago

see also #138

g-k commented 4 years ago

OK I landed a benchmark for https://github.com/mozilla-services/autograph/issues/347#issuecomment-539553583 in #397, but we might not have the right signing options:

https://github.com/mozilla-services/autograph/pull/397/files#diff-1a006257a05175ebd800c58603fd43eeR42-R46

@Callek you're using the SHA256 PKCS7 digest and COSE ES256 right? Are there any other options you're passing?

Callek commented 4 years ago

@Callek you're using the SHA256 PKCS7 digest and COSE ES256 right? Are there any other options you're passing?

Correct, only those:

        sign_req["options"]["cose_algorithms"] = ["ES256"]
        sign_req["options"]["pkcs7_digest"] = "SHA256"

(from: https://github.com/mozilla-releng/scriptworker-scripts/blob/326436e6e622fd8dd70e8985edbf1d0319ddd5ab/iscript/src/iscript/autograph.py#L247)

@escapewindow may have a different algorithm in use for something else, but I'm not seeing it if so.