Performance characterisation for ALS - Setup test-cases

[PaulGregoryBaker]

Goal:

As a Mojaloop Hub adopter I want to know the isolated performance characterisation of the ALS
so that I can design an environment that depends on it's output.

Acceptance Criteria:

• [x] Verify that the transactions per second and the parallelisation is specified / identified for this test.
• [x] Verify that K6 tests scripts are written to run against ALS
  • [x] Outgoing requests have trance headers with that start timestamp
• Create a new service to receive call-backs and process the trace headers (capture metrics in some store (Influx, Prometheus etc...). Support callbacks for both parties and participants.
  • Calculate metrics and store metrics into timeseries database
• Create call-back rule for TTK
• [x] Decide on test set (caching)
  • [x] GET parties test
  • GET participants test
• [x] Verify that a baseline performance test can be run by running the test-harness without the ALS
• Add Grafana dashboard to be able to show the end to end transfer to generate baseline

Note Depending on how this story plays out, there may be an opportunity to scale out to larger deploy and run more comprehensive tests.

The following will need to be mocked :

1. Central ledger
2. Oracle
3. Get endpoints
4. Payee DFSP (need to decide how to do this)

Considering using the TTK for these Mocks; if we do then will need to characterise the TTK.

SDK Consider using the SDK for this, as the SDK would be a more realistic test. If this is used, then the SDK will too need to be characterised. SDK will need to be scaled for this to work.

Complexity: <High|Medium|Low> > A short comment to remind the reason for the rating

Uncertainty: <High|Medium|Low> > A short comment to remind the reason for the rating

---

Tasks:

• [x] Define the approach for using k6 to test different scenarios
• [x] Workshop Test-cases for ALS Performance Characterization
  • [x] Define Test-cases
  • [x] Review Test-cases
• [x] Write Test-cases in k6 scripts based on approach
• [x] Test Test-cases against Test-Harness
• [x] Characterize K6 Test Cases against Callback Handler Service (i.e. without ALS) for a baseline.

Done

• [x] Acceptance Criteria pass
• [x] Designs are up-to date
• [x] Unit Tests pass
• [x] Integration Tests pass
• [x] Code Style & Coverage meets standards
• [x] Changes made to config (default.json) are broadcast to team and follow-up tasks added to update helm charts and other deployment config.

Pull Requests:

• https://github.com/mojaloop/ml-core-test-harness/pull/15

Follow-up:

• N/A

Dependencies:

• N/A

Accountability:

• Owner: TBC
• QA/Review: TBC

[PaulGregoryBaker]

Hey team! Please add your planning poker estimate with Zenhub @vijayg10 @mdebarros

[elnyry-sam-k]

Two points on this from me, @PaulGregoryBaker

1. Mocking only the Payee DFSP makes sense to me (the rest are part of the ALS as a system (might not be as a service); I've made a comment on another similar issue)
2. It might make sense to have a separate issue "to first run a single ALS call end to end with JMeter and related services" and then get into baselining , TPS expectations etc.. the Setup can involve tasks for mocks, JMeter setup, handling callbacks and such others..

[PaulGregoryBaker]

Please add your planning poker estimate with Zenhub @kleyow

[mdebarros]

Test Cases (Draft)

Scenario: FSPIOP Participants

1. POST /participants/{Type}/{ID} happy path
   1. Static Type and ID
2. POST /participants/{Type}/{ID}/{SubId} happy path
   1. Static Type and ID
3. POST /participants/{Type}/{ID} with error callback
   1. OpenAPI Validation Failure
   2. Connectivity Timeout
4. POST /participants/{Type}/{ID}/{SubId} with error callback
   1. OpenAPI Validation Failure
   2. Connectivity Timeout

Scenario: FSPIOP Parties

1. POST /parties/{Type}/{ID} happy path
   1. Static Type and ID
2. POST /parties/{Type}/{ID}/{SubId} happy path
   1. Static Type and ID
3. POST /parties/{Type}/{ID} with error callback
   1. OpenAPI Validation Failure
   2. Connectivity Timeout
4. POST /parties/{Type}/{ID}/{SubId} with error callback
   1. OpenAPI Validation Failure
   2. Connectivity Timeout

Scenario: FSPIOP Mix

1. Combination of POST /parties/ and POST /participants/ Scenarios of both Happy and Eroor callback Test Case Scenarios

Scenario: Admin

1. POST /oracles
2. GET /oracles/{ID}
3. PUT /oracles/{ID}
4. DELETE /oracles/{ID}
5. Mix of POST, GET, PUT, DELETE /oracles/*

[elnyry-sam-k]

Thanks @mdebarros for the details.

Looks like a typo in the Scenario: FSPIOP Mix section POST /parties/* ?