DiamondLightSource / mx-bluesky

Bluesky plans, plan stubs, and utilities for MX beamlines
https://diamondlightsource.github.io/mx-bluesky/
Apache License 2.0
0 stars 2 forks source link

Murko: Investigate ways of storing image data in redis #592

Open DominicOram opened 1 month ago

DominicOram commented 1 month ago

As of https://github.com/DiamondLightSource/dodal/pull/809 we store image data as jpeg bytes. However, we could use something better optimised for binary like https://redis.io/docs/latest/develop/data-types/bitfields/

We should evaluate based on minimising the storage requirements on the DB but without adding the need for any complex code for storing/retrieving (e.g. should still be possible in a couple of lines)

Acceptance Criteria

DominicOram commented 1 month ago

From a bit more of an indepth look it looks like bitfields are more optimized for scenarios where you want to bit-wise manipulation on data e.g. storing a large set of true/false values in a single byte string. This is not really what we want but if it does provide optimisation on the space then open to trying it