tc400 agent throws error when `turn_turbo_off` task is run

[msilvafe]

@ngalitzki noticed when he ran the turn_turbo_off task on the ocs-web interface an error was thrown and the session data for the task says "Turbo unable to be turned off" but the turbos appear to shut off just fine. Seems like a logging/agent error.

[BrianJKoopman]

Interesting. It would be useful if there was a time we could debug this, which would involve turning the turbo on and off.

I'd like to see what the actual responses are to this line, if not at a lower level: https://github.com/simonsobs/socs/blob/5899c242298a86d8539d5cfd155c1f071ffc8d0b/socs/agents/pfeiffer_tc400/drivers.py#L178

We do have a test for this task with mock communication, so either we have our expected responses wrong, or the TC400 is sending something unexpected.

[ngalitzki]

Running another LC set over the weekend, probably warm up sometime next week. Once the mix is fully recovered we can turn the turbos on and off to our hearts content.

[mjrand]

I second the idea that the turbo is sending something unexpected back. I feel like we saw this issue once before at UCSD a long time ago where the agent would unexpectedly crash because the turbo responded something weird during acq. I could never consistently reproduce the error, but maybe we'll be able to reproduce this if we turn it on and off enough next week.

[ngalitzki]

@BrianJKoopman mix is recovered and turbos are in a good vacuum so now is a good time to debug this.

[sanahabhimani]

I pinged Michael separately, but just wanted to note that whenever there's another good time to test, feel free to holler at me if Brian's too busy.

[kmharrington]

Running into this again at the site. Would be nice to not have to worry about getting the turbos off.

[BrianJKoopman]

I still need the details to that device response above. Lemme add some debug lines and we can run that at site so I can get the info I need from it next time this happens.

[jlashner]

ya for now we can just add a more informative error message