search

bcgov / aries-load-generator

A load generator service to test Aries agents
Apache License 2.0
0 stars 2 forks source link

Load testing verifications #8

Open esune opened 9 months ago

esune commented 9 months ago

Review the endpoints exposed by VC-AuthN so that it is possible to load test the system by generating verification events without having to go through the UI/QR Code.

swcurran commented 9 months ago

Let’s do this in stages, starting from easiest first. Here’s what I propose. We'll should do the first one now driven by this issue, and based on the results, we’ll look at doing the other steps (2, 3 below) as new issues:

  1. Copy the existing “Verify” script (locustMediatorPresentProof.py) to a new script (e.g., locustMediatorVerifier.py) and adjust it by moving the “get credential” step out of the loop and into the setup, so that we are only doing verifies in the loop. We’ll run that with Traction Dev and see how we do in the loop step. The credential can be trivial for this — just the default one with the single attribute. The goal is to see how many presentations the Traction Verifier can process per minute.

Notes:

The rest of this will be in different tickets if we decide we need to do them. Included here just to outline the direction.

  1. Make a new “IAS” version of the script, where we replace the “generic issuer” get credential in the start up with the IAS “get person credential” processing. We still would just do the get credential once, and loop on the verifications, but we would be using a much bigger credential.

  2. We add a way to use in an instance of VC-Authn as the verifier. I think it can be done be deploying the version of VC-Authn app with the “deep link” on it, and in the loop “curl”-ing that URL, extracting the invitation from the deep-link URL (we don’t actually call the deep-link — we just extract the base64-encoded invitation and process it). I think that should result in the verification being done.

loneil commented 8 months ago

For item 1 above:

Used the Sandbox env for these tests and created 2 tenants. Can repeat with different envs and timeframes if we want.

Issuer 4d67bef7-4da4-480b-8f5a-db2147bfdb7b a393bc93-0c60-4728-983e-2a387f62cc52

Copied in the schema used previously (LjFT93n4TcoUZZ8b5CcwMp:2:load_testing_prefs:1.0) and created a credential and a revocable credential from it.

Verifier 6f1f24f7-031a-42f8-95bf-17793be07f18 14319a1b-b767-4239-84a8-7caf3df9e475

Connection with the verifier? Yes existing locustMediatorPresentProof scripts use a connection between the local agent and the Traction verifier and then that connection ID is used in the call to request verification. The step that does the verification is invoking the Traction /present-proof/send-request endpoint with the connection ID

Akrida changes Based locustMediatorVerifier.py off of locustMediatorPresentProof.py but the Akrida code seems to have a bunch of copy/paste errors (I think?) in the script, client, and agent code where the issuer agent and config is being used for verifier stuff. So I've adjusted that locally and will make a PR.

In the script start up 1-time step does

Then the repeating presentation_exchange step invokes the verifier Tratction agent /present-proof/send-request

Sample run Ran this through Sandbox, just did an arbitrary 1 minute manual timed start/stop (some of that time is taken up with startup connection/cred steps)

Did this twice, once for the non-revocable cred, once with the revocable.

8WDFumYo5sd2n9Zo1mPJoK:3:CL:249336:default image

8WDFumYo5sd2n9Zo1mPJoK:3:CL:249336:default_rev (also happened to land on 88 requests but that's mostly luck depending on when I hit stop) image

Again through Tenant UI (Verifier tenant) can see all the many verifications.

loneil commented 8 months ago

For item 2 (use person credential):

As discussed just copied the IDIM person credential to one of my own and issue that from Sandbox issuer. Make the same schema as the IDIM person cred, but on BCOVRIN-TEST LCpcJGQ3n3HKoYBbEHh6JL:2:Person:1.0

Make 2 creds on Sandbox Issuer tenant CRED_DEF=8WDFumYo5sd2n9Zo1mPJoK:3:CL:405243:person CRED_DEF=8WDFumYo5sd2n9Zo1mPJoK:3:CL:405243:person_rev

Set CRED_ATTR to the values as shown:

[ { "name": "street_address", "value": "123 FAKE STREET" }, { "name": "country", "value": "CA" }, { "name": "locality", "value": "CITYTOWN" }, { "name": "birthdate_dateint", "value": "20011222" }, { "name": "given_names", "value": "TEST" }, { "name": "region", "value": "BC" }, { "name": "postal_code", "value": "V0H 1V0" }, { "name": "family_name", "value": "USER" }, { "name": "picture", "value": "data:image/jpeg;base64,BASE64GOESHERE...}, { "name": "expiry_date_dateint", "value": "" } ]

Used this image (in place of BASE64GOESHERE above) which is what the IAS controller gives when issuing: image The raw base64 text chars used take up about 12KB (should use a 'real' image? bigger?)

Same pattern as above (start, time one minute, stop), do non-revocable, then revocable.

CRED_DEF=8WDFumYo5sd2n9Zo1mPJoK:3:CL:405243:person image

CRED_DEF=8WDFumYo5sd2n9Zo1mPJoK:3:CL:405243:person_rev image

A little slower on the average, but not orders of magnitude or anything, possibly just environmental. Can refine with less abitrary test runs later.

swcurran commented 8 months ago

Trying to set up on Test. I’m using “stock” Akrida with the .env updated (I think) appropriately, and the new “Verifier” script added. None of the “load-agent” changes (locustAgent, agent.ts) from the IAS testing remain — is that right?

When I run it, the "on start” task goes fine (60 with 0 errors), all agents start successfully, but all of the verifications fail. Really fast fails (so there is that), but they all fail (384 of them)

In the Locust UI I get: # lookup in project for bcovern CRED_ATTR='[{"mime-type": "text/plain","name": "score","value": "test"}]' SCHEMA="LjFT93n4TcoUZZ8b5CcwMp:2:load_testing_prefs:1.0" # anchor schema VERIFIED_TIMEOUT_SECONDS=60 # seconds for verified: true

In the docker compose log, I get a bunch of these — although not 384 of them:

load-agent-1  | (node:16) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 ProofStateChanged listeners added to [EventEmitter]. Use emitter.setMaxListeners() to increase limit
load-agent-1  | (Use `node --trace-warnings ...` to show where the warning was created)
load-agent-1  | DEBUG: Verifier Connection: {'_tags': {'state': 'initial', 'role': 'receiver', 'recipientKeyFingerprints': ['z6Mkfqh7zQKy6YuNDvbu5HEeNuaBpncATVknhdZhGqEcm7nS'], 'threadId': 'ef576547-03ad-408c-af21-dcccc8286ae8', 'invitationId': 'ef576547-03ad-408c-af21-dcccc8286ae8'}, 'metadata': {'_internal/legacyInvitation': {'legacyInvitationType': 'connections/1.x'}}, 'id': 'da37ff6b-dc88-467c-9d56-51faac3ab44d', 'createdAt': '2024-03-01T19:38:35.017Z', 'outOfBandInvitation': {'@type': 'https://didcomm.org/out-of-band/1.1/invitation', '@id': 'ef576547-03ad-408c-af21-dcccc8286ae8', 'label': 'Test', 'accept': ['didcomm/aip1', 'didcomm/aip2;env=rfc19'], 'handshake_protocols': ['https://didcomm.org/connections/1.0'], 'services': [{'id': '#inline', 'serviceEndpoint': 'https://traction-acapy-test.apps.silver.devops.gov.bc.ca', 'type': 'did-communication', 'recipientKeys': ['did:key:z6Mkfqh7zQKy6YuNDvbu5HEeNuaBpncATVknhdZhGqEcm7nS']}]}, 'role': 'receiver', 'state': 'prepare-response', 'autoAcceptConnection': True, 'reusable': False, 'updatedAt': '2024-03-01T19:38:35.022Z'}

Perhaps we can do a debug session later?

swcurran commented 8 months ago

Issue above was resolved. Another test run with some questions (see #9 comments).

Not sure we want to bother, since the processing would be about the same, but for future consideration if we decide to do more:

I would suggest that we drop the verifier connection establishment in the "on_start", and instead use do a connection-less presentation request in the loop. That would be more like what would happen in, for example, a VC-Authn example.