simonsobs / socs

Simons Observatory specific OCS agents.
BSD 2-Clause "Simplified" License
12 stars 12 forks source link

HWP Emulation and PID agent lockless restructure #606

Closed jlashner closed 5 months ago

jlashner commented 6 months ago

This PR begins to implement an emulation framework for the HWP, to be used for testing HWP agent operation. Also restructures the HWP_PID to use the lockless format.

Description

During the dev call, and while testing out HWP supervisor status at the site, there are a number of things that we still need to update in many of the HWP agents before we can reliably use them to observe. Due to the limited access to HWP hardware, a relatively realistic HWP emulator will be extremely helpful for developing changes, and testing them before testing on real hardware. This PR begins to implement this emulation, and I have tested it running against PID and PMX agents on my local development setup.

The data being returned from the emulator isn't perfect (especially since the PID interface requires some more reverse engineering), but it is good enough such that agent tasks are able to run successfully.

In testing, I was finding many instances in which the PID agent could not successfully run operations due to the locking structure, so I refactored that agent to have the same structure as the new timeout-lockless agents, and tested with the emulator.

This PR contains the following changes:

How Has This Been Tested?

Tested by running with the HWP PMX and PID agents locally.

Types of changes

Checklist:

jlashner commented 5 months ago

Hi Brian, thanks for the comments, I think I addressed or responded to all of them.

My goal with this is not necessarily to use this in tests, though it may be possible. Mainly the purpose of this is to allow myself and other HWP developers to run HWP agents for the PID, PMX, supervisor, etc. on their local systems, which is very helpful for developing new features and testing behavior.

jlashner commented 5 months ago

Hi Brian, so I wasn't originally intending to use the emulator for tests, but when trying to fix the tests for the PID agent restructure, it ended up being much easier to use the emulator, since the PID responses are difficult to write out manually.

With the emulator, the tests work well though so I moved it to the socs.testing sub-package. Let me know if this is still ok with you.