simonsobs / scheduler

Scheduler for SO
0 stars 1 forks source link

Moon scan runs into satp1 software prelimit for az CCW sector #90

Open ktcrowley opened 3 weeks ago

ktcrowley commented 3 weeks ago

I'm not sure this is a common fault in calibration obs or point source obs, but we had a surprise ACU agent alarm during a generate_scan command. The full schedule is here: https://site.simonsobs.org/schedules/satp1/2024-08-20T20%3A00%3A00%2B00%3A00_2024-08-21T20%3A00%3A00%2B00%3A00.py

Relevant commands in block below:

  now = datetime.datetime.now(tz=UTC)
  scan_start = datetime.datetime(2024, 8, 21, 1, 54, 17, 884970, tzinfo=datetime.timezone.utc)
  scan_stop = datetime.datetime(2024, 8, 21, 4, 58, 40, 592471, tzinfo=datetime.timezone.utc)
  if now > scan_start:
      # adjust scan parameters
      az = 439.996 + -0.00423*(now-scan_start).total_seconds()
  else: 
      az = 439.996
  if now > scan_stop:
      # too late, don't scan
      pass
  else:
      run.acu.move_to(az, 50.0)

      print('Waiting until 2024-08-21 01:54:17.884970+00:00 to start scan')
      run.wait_until('2024-08-21T01:54:17.884970+00:00')

      run.seq.scan(
          description='moon', 
          stop_time='2024-08-21T04:58:40.592471+00:00', 
          width=28.157, 
          az_drift=-0.00423, 
          subtype='cal',
          tag='moon,rising,ws1,ws0,ws4',
      )
      run.smurf.bias_step(concurrent=True)

I'm not sure where the safe az range is specified in the satp1.py policy file on simple-sun branch, but it seems it may not be properly set. Our HW limits are -90 to 450, but we want to program a safe distance away from the software pre-limits, which define a narrower range. Let me know if these numbers are needed.

kmharrington commented 3 weeks ago

@ktcrowley is this a schedule you generated or one of mine? The software az limit has been set at 405 for awhile so I'm very confused how that got in. https://github.com/simonsobs/scheduler/blob/e68c22a93cd6c5d056c4a322a119eca2b8fcbb9c/src/schedlib/policies/satp1.py#L240

kmharrington commented 2 weeks ago

ok yeah, this came from mine. @mhasself @guanyilun BIG issue in the simple-sun branch. I apparently left the moon scan on az_branch: 270 without noticing and it allowed me to produce a scan that went outside our stated Azimuth limits. This really should not be allowed to be possible.

Config file the reproduces the error

t0: 2024-08-20T20:00:00+00:00
t1: 2024-08-21T20:00:00+00:00
t0_state_file: None

elevation: 60
boresight: 0
az_speed: 0.5
az_accel: 1.0

hwp_dir: True
stow_at_end: True
run_relock: False

cal_targets:
  - source: moon
    # boresight: # if not specified, use scan boresight
    elevation: 50
    focus: 'ws0,ws4,ws1'
    allow_partial: False
    az_branch: 270
  - source: jupiter
    # boresight: # if not specified, use scan boresight
    elevation: 48
    focus: 'ws1'
    allow_partial: True
    az_branch: 270