frmscoe / General-Issues

This repo exists to track current work and any issues within the FRMS CoE
2 stars 0 forks source link

Performance hypothesis: Protocol buffers for Redis cache interaction - TADProc #265

Closed Justus-at-Tazama closed 10 months ago

Justus-at-Tazama commented 10 months ago

Ref: https://frmscoe.atlassian.net/wiki/spaces/FRMS/pages/68485132/Protobuf+vs+Json Ref: https://protobuf.dev/

As an Actio platform operator, I want interaction with Redis cache to be as efficient as possible, So that I can maximise performance at minimal cost.

Acceptance criteria:

image

  1. For each of the following platform processors, replace the serialisation of the Redis value object into JSON prior to writing the cache with a protocol buffer:

    • TMS API (Data Preparation Step)
    • Typology Processor (Interim results cache)
    • CADProc (Interim results cache)
    • TADProc (Interim results cache)
  2. For each of the following platform processors, replace the deserialisation of the Redis value object from JSON after reading from the cache with a protocol buffer:

    • TMS API (Data Preparation Step)
    • Typology Processor (Interim results cache)
    • CADProc (Interim results cache)
    • TADProc (Interim results cache)