simonsobs / scheduler

Scheduler for SO
0 stars 1 forks source link

Error related with sun-safe parking spot. #83

Open yoshinori-0778 opened 3 months ago

yoshinori-0778 commented 3 months ago

When I make schedule including jupiter scan, I have an error below.

Scripts I am using are for satp3, but this problem should be common.

(so-base-py310) ysueno@login:~/workspace/script/scheduler-scripts/satp3$ python3 TSATScheduler.py 2024-08-13 --source jupiter --elevation 48 --focus ws6
2024-08-13 19:28:48,174 [INFO] applying sun avoidance rule: {'min_angle': 49, 'min_sun_time': 1980}
2024-08-13 19:28:50,792 [INFO] planning calibration scans...
2024-08-13 19:28:50,792 [INFO] -> planning calibration scans for CalTarget(source='jupiter', array_query='ws6', el_bore=48, tag='ws6', boresight_rot=0, allow_partial=True, drift=True, az_branch=270)...
2024-08-13 19:28:51,171 [INFO] -> allow_partial = True: trimming scan options by sun rule
2024-08-13 19:28:51,953 [INFO] -> found 4 scan options for jupiter (ws6): [
  ScanBlock(jupiter, 24-08-13 09:49:08 -> 24-08-13 12:03:12, az=377.49, el=48.00, throw=16.00, drift=-0.00470)),
  ScanBlock(jupiter, 24-08-13 12:03:07 -> 24-08-13 14:17:04, az=334.62, el=48.00, throw=16.01, drift=-0.00460)),
  ScanBlock(jupiter, 24-08-14 09:45:57 -> 24-08-14 11:59:58, az=377.46, el=48.00, throw=16.00, drift=-0.00470)),
  ScanBlock(jupiter, 24-08-14 11:59:50 -> 24-08-14 14:13:43, az=334.63, el=48.00, throw=16.01, drift=-0.00460))
]
2024-08-13 19:28:51,953 [INFO] applying calibration policy - round-robin - to resolve calibration target conflicts
2024-08-13 19:28:52,306 [INFO] -> after calibration policy: [
  ScanBlock(jupiter, 24-08-13 09:49:08 -> 24-08-13 12:03:07, az=377.49, el=48.00, throw=16.00, drift=-0.00470)),
  ScanBlock(jupiter, 24-08-13 12:03:07 -> 24-08-13 14:17:04, az=334.62, el=48.00, throw=16.01, drift=-0.00460)),
  ScanBlock(jupiter, 24-08-14 09:45:57 -> 24-08-14 11:59:50, az=377.46, el=48.00, throw=16.00, drift=-0.00470)),
  ScanBlock(jupiter, 24-08-14 11:59:50 -> 24-08-14 14:13:43, az=334.63, el=48.00, throw=16.01, drift=-0.00460))
]
2024-08-13 19:28:52,656 [INFO] applying min duration rule: {'min_duration': 600}
2024-08-13 19:28:52,657 [INFO] ================ pass 1 ================
2024-08-13 19:28:52,657 [INFO] step 1: planning pre-session ops
2024-08-13 19:28:52,658 [INFO] step 2: planning calibration scans
2024-08-13 19:28:52,770 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-13 09:49:08 -> 24-08-13 12:03:07, az=377.49, el=48.00, throw=16.00, drift=-0.00470))
2024-08-13 19:28:52,780 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-13 12:03:07 -> 24-08-13 14:17:04, az=334.62, el=48.00, throw=16.01, drift=-0.00460))
2024-08-13 19:28:52,853 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-14 09:45:57 -> 24-08-14 11:59:50, az=377.46, el=48.00, throw=16.00, drift=-0.00470))
2024-08-13 19:28:52,863 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-14 11:59:50 -> 24-08-14 14:13:43, az=334.63, el=48.00, throw=16.01, drift=-0.00460))
2024-08-13 19:28:52,877 [INFO] step 3: planning cmb ops
2024-08-13 19:28:53,010 [INFO] step 4: planning post-session ops
2024-08-13 19:28:53,011 [INFO] not enough time for post-session operations, trimming...
2024-08-13 19:28:53,011 [INFO] ================ pass 2 ================
2024-08-13 19:28:53,011 [INFO] step 1: planning pre-session ops
2024-08-13 19:28:53,012 [INFO] step 2: planning calibration scans
2024-08-13 19:28:53,029 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-13 09:49:08 -> 24-08-13 12:02:07, az=377.49, el=48.00, throw=16.00, drift=-0.00470))
2024-08-13 19:28:53,034 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-13 12:15:07 -> 24-08-13 14:16:04, az=331.31, el=48.00, throw=16.01, drift=-0.00460))
2024-08-13 19:28:53,106 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-14 09:45:57 -> 24-08-14 11:58:50, az=377.46, el=48.00, throw=16.00, drift=-0.00470))
2024-08-13 19:28:53,112 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-14 12:11:50 -> 24-08-14 14:12:43, az=331.32, el=48.00, throw=16.01, drift=-0.00460))
2024-08-13 19:28:53,126 [INFO] step 3: planning cmb ops
2024-08-13 19:28:53,254 [INFO] step 4: planning post-session ops
2024-08-13 19:28:53,255 [INFO] ================ pass 3 ================
2024-08-13 19:28:53,255 [INFO] step 1: planning pre-session ops
2024-08-13 19:28:53,255 [INFO] step 2: planning calibration scans
2024-08-13 19:28:53,272 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-13 09:49:08 -> 24-08-13 12:01:07, az=377.49, el=48.00, throw=16.00, drift=-0.00470))
2024-08-13 19:28:53,277 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-13 12:15:07 -> 24-08-13 14:15:04, az=331.31, el=48.00, throw=16.01, drift=-0.00460))
2024-08-13 19:28:53,352 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-14 09:45:57 -> 24-08-14 11:57:50, az=377.46, el=48.00, throw=16.00, drift=-0.00470))
2024-08-13 19:28:53,357 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-14 12:11:50 -> 24-08-14 14:11:43, az=331.32, el=48.00, throw=16.01, drift=-0.00460))
2024-08-13 19:28:53,371 [INFO] step 3: planning cmb ops
2024-08-13 19:28:53,495 [INFO] step 4: planning post-session ops
2024-08-13 19:28:53,495 [WARNING] round_trip: ir did not converge after 3 passes
2024-08-13 19:28:53,496 [INFO] ================ lowering ================
2024-08-13 19:28:53,496 [INFO] step 1: planning pre-session ops
2024-08-13 19:28:53,496 [INFO] step 2: planning calibration scans
2024-08-13 19:28:53,512 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-13 09:49:08 -> 24-08-13 12:00:07, az=377.49, el=48.00, throw=16.00, drift=-0.00470))
2024-08-13 19:28:53,517 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-13 12:15:07 -> 24-08-13 14:14:04, az=331.31, el=48.00, throw=16.01, drift=-0.00460))
2024-08-13 19:28:53,585 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-14 09:45:57 -> 24-08-14 11:56:50, az=377.46, el=48.00, throw=16.00, drift=-0.00470))
2024-08-13 19:28:53,590 [INFO] -> planning cal block: ScanBlock(jupiter, 24-08-14 12:11:50 -> 24-08-14 14:10:43, az=331.32, el=48.00, throw=16.01, drift=-0.00460))
2024-08-13 19:28:53,603 [INFO] step 3: planning cmb ops
2024-08-13 19:28:53,726 [INFO] step 4: planning post-session ops
2024-08-13 19:28:53,726 [INFO] ================ solve moves ================
2024-08-13 19:28:53,726 [INFO] step 1: solve sun-safe moves
Traceback (most recent call last):
  File "/so/home/ysueno/workspace/script/scheduler-scripts/satp3/TSATScheduler.py", line 37, in <module>
    seqs = policy.build_schedule(t0, t1)
  File "/so/home/ysueno/workspace/script/scheduler/src/schedlib/policies/sat.py", line 769, in build_schedule
    ir = self.seq2cmd(seqs, t0, t1, state)
  File "/so/home/ysueno/workspace/script/scheduler/src/schedlib/policies/sat.py", line 714, in seq2cmd
    ops, state = build_op.apply(seq, t0, t1, state, self.operations)
  File "/so/home/ysueno/workspace/script/scheduler/src/schedlib/policies/stages/build_op.py", line 131, in apply
    ir = PlanMoves(**self.plan_moves).apply(ir)
  File "/so/home/ysueno/workspace/script/scheduler/src/schedlib/policies/stages/build_op.py", line 693, in apply
    gaps = get_safe_gaps(seq[i-1], seq[i])
  File "/so/home/ysueno/workspace/script/scheduler/src/schedlib/policies/stages/build_op.py", line 681, in get_safe_gaps
    raise ValueError("Next scan not accessible from sun-safe parking spot.")
ValueError: Next scan not accessible from sun-safe parking spot.
kmharrington commented 2 months ago

Going to add another example here where SATp1 hit a sun safety error while setting up a Jupiter scan on the simple-sun branch.

This error: https://site.simonsobs.org/satp1/nextline/db/runs/1321

Can be reproduced with the configuration below (except the offending scan is on line 379 instead of 349, difference just from now passing a state file).

# yaml loads iso format automatically into datetimes
t0: 2024-08-19T20:00:00+00:00
t1: 2024-08-20T20:00:00+00:00
t0_state_file: None

elevation: 60
boresight: 0
az_speed: 0.5
az_accel: 1.0

hwp_dir: True
stow_at_end: False
run_relock: False

cal_targets:
  - source: saturn
    # boresight: # if not specified, use scan boresight
    elevation: 50
    focus: 'ws4'
    allow_partial: False
  - source: jupiter
    # boresight: # if not specified, use scan boresight
    elevation: 48
    focus: 'ws1,ws6'
    allow_partial: True
    az_branch: 270 
  - source: jupiter
    # boresight: # if not specified, use scan boresight
    elevation: 48
    focus: 'ws1,ws2'
    allow_partial: True
    az_branch: 270 
mhasself commented 2 months ago

The error Katie reports here is actually a problem in the ACU agent. See https://github.com/simonsobs/socs/issues/744.

Yoshi's error still requires investigation.