search

tazama-lf / General-Issues

This repo exists to track current work and any issues within the FRMS CoE
2 stars 0 forks source link

Performance testing - Distributed Redis keys #70

Open Justus-at-Tazama opened 1 year ago

Justus-at-Tazama commented 1 year ago

Performance Hypothesis

All Redis cache interaction is consolidated into a single database for all processors. If each processor "tranche" can have access to its own Redis cache database, performance will improve.

Testing approach:

  1. For each of the following group of processors, implement a separate Redis cache database:

    a. Data Preparation Service (message-set and relationships pre-caching) - #1 b. Channel router and setup processor (network map retrieval) c. Rule processors (rule configuration retrieval) d. Typology processor(s) (typology configuration retrieval and typology completion management) e. Channel Aggregation and Decisioning Processor(s) (channel completion management) f. Transaction Aggregation and Decisioning Processor(s) (transaction completion management)

  2. Re-run previous performance test ingestion pack.

  3. Compare average and peak throughputs (TPS) and average turnaround time per transaction (API ingestion to evaluation result posting in milliseconds) results to previous benchmark - #69

Justus-at-Tazama commented 5 months ago

Blocked due to recent other Redis optimizations removing the need for this enhancement.