Make a redis lookup table caching device

stan-dot commented 1 month ago

As per discussion with @callumforrester and @DiamondJoseph this is needed for db IO as this cannot be in a plan or a stub as it would clog the RunEngine

problem description

i18 lookup tables are saved in brittle csv documents, with manual steps as part of the alignment process.

proposed solution

add a LookupTable device as part of the Undulator or Undulator-DCM system. The device holds a connection to a key value fast lookup store (like Redis, already deployed at i18). Alignment is done automatically just thorugh specifying the desired element and edge, without manual steps. At first peak matching is supported, later a weighed average algorithm could be added as an option.

relevant beamlines

i10 (@Relm-Arrowny has recently done work on that https://github.com/DiamondLightSource/dodal/pull/744), and MX i03, i04 (as discussed with @DominicOram ) at the moment have ophyd-async devices that read the lookup tables from the filesystem

Acceptance Criteria

I can spin the device up at i18 and make a quick test plan to test redis read and write

callumforrester commented 1 month ago

@stan-dot do you plan to put an API between the device and redis?

stan-dot commented 3 weeks ago

what do you mean? I'd use the redis client library and point at the redis instance on the cluster. which would need an ingress

stan-dot commented 3 weeks ago

todo: need to move the temporary implementation from the i18-bluesky repo

stan-dot commented 1 week ago

as @DominicOram commented elsewhere it fits best to include this as a child device in Undulator.

also this is relevant https://github.com/DiamondLightSource/i18-bluesky/pull/14

stan-dot commented 1 week ago

as the scan also uses data from the diode, it remains to be decided how to split the logic between the plan and the device. we could have a data structure to get from the diode, and do all the calculations from the device

DiamondLightSource / dodal