simonsobs / sorunlib

High level library for running observatory operations using OCS.
BSD 2-Clause "Simplified" License
0 stars 0 forks source link

Create function to check if operations start properly #119

Closed BrianJKoopman closed 10 months ago

BrianJKoopman commented 10 months ago

We've seen some errors recently where acu.generate_scan doesn't run properly within a seq.scan call. This PR introduces an internal check_started function, similar to the other "check" functions, which is meant to be called immediately after a <process>.start() call is made. It'll wait for the operation state to go from 'starting' to 'running', and raise an exception if that doesn't happen.

For SMuRFs starting their 'stream' process, if this occurs that SMuRF will be removed from the targets list, much like if a single task were to fail for a given SMuRF, up to the configured threshold of failures.

Resolves #56.