search

sot / starcheck

BSD 3-Clause "New" or "Revised" License
3 stars 0 forks source link

Minimal change to default to flight scenario for commands v2 #391

Closed taldcroft closed 2 years ago

taldcroft commented 2 years ago

Description

For production use of starcheck (run by SOT MP on HEAD), we should use the flight scenario by default so that it does not use external network resources. In particular using OCCweb requires credentials that may not be available.

This uses on_head_network() to decide to default to flight.

Interface impacts

Testing

Unit tests

Functional tests

On HEAD (kady)

ska3-kady$ env KADI_COMMANDS_VERSION=2 ./sandbox_starcheck -dir ~/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls
Using backstop file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/CR225_1108.backstop
Using guide summary file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/mps/mgAUG1322A.sum
Using OR file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/mps/or/AUG1322_A.or
Using maneuver file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/mps/mmAUG1322A.sum
Using DOT file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/mps/mdAUG1322A.dot
Using mech check file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/output/TEST_mechcheck.txt
Using fidsel file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/History/FIDSEL.txt
Using dither file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/History/DITHER.txt
Using radmon file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/History/RADMON.txt
Using simtrans file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/History/SIMTRANS.txt
Using simfocus file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/History/SIMFOCUS.txt
Using attitude file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/History/ATTITUDE.txt
Using characteristics file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/mps/ode/characteristics/CHARACTERIS_18JAN21
Using aimpoint file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/output/AUG1322A_dynamical_offsets.txt
Using config file /data/baffin/tom/git/starcheck/starcheck/data/characteristics.yaml
Using odb file /data/baffin/tom/git/starcheck/starcheck/data/fid_CHARACTERISTICS
Using agasc_file file /proj/sot/ska3/flight/data/agasc/proseco_agasc_1p7.h5
Using manerr file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/output/AUG1322A_ManErr.txt
Using processing summary file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/mps/msAUG1322A.sum
Using TLR file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/CR225_1108.tlr
Using banned_agasc file /data/baffin/tom/git/starcheck/starcheck/data/agasc.bad
Using bad_pixel file /data/baffin/tom/git/starcheck/starcheck/data/ACABadPixels
Using acq_star_rdb file /data/baffin/tom/git/starcheck/starcheck/data/bad_acq_stars.rdb
Using gui_star_rdb file /data/baffin/tom/git/starcheck/starcheck/data/bad_gui_stars.rdb
Dither status in kadi commands does not match DITHER history
Read 258 ACA bad pixels from /data/baffin/tom/git/starcheck/starcheck/data/ACABadPixels
Read 46 bad AGASC IDs from /data/baffin/tom/git/starcheck/starcheck/data/agasc.bad
#####################################################################
# calc_ccd_temps run at Mon Aug 29 06:39:37 2022 by aldcroft
# Continuity run_start_time = 2022:241:10:39:37.159
# calc_ccd_temps version = 13.15.2.dev7+ge7c6691
# chandra_models version = 3.40.2
# kadi version = 6.0.1
#####################################################################

Using backstop file /home/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/CR225_1108.backstop
Found 2255 backstop commands between 2022:225:11:37:00.000 and 2022:234:01:26:00.000
RLTT = 2022:225:11:37:00.000
sched_stop = 2022:234:01:26:00.000
Fetching telemetry between 2022:224:11:37:00.000 and 2022:225:11:37:00.000
Getting commands from archive only
Loaded /proj/sot/ska3/flight/data/kadi/cmds2.h5 with 1418496 commands
Getting commands from archive only
Getting commands from archive only
Getting commands from archive only
Calculating ACA thermal model
Propagation initial time and ACA: 2022:225:11:21:50.816 -10.37
Making temperature check plots
Writing plot file starcheck/ccd_temperature.png
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int' at line 45
 at ./starcheck/src/starcheck.pl line 37.
    main::__ANON__("TypeError: unsupported operand type(s) for +: 'NoneType' and "...) called at (eval 79) line 3
    Ska::Starcheck::Obsid::proseco_probs("__main__", "proseco_probs", HASH(0x55f98d26c3e0)) called at starcheck/src/lib/Ska/Starcheck/Obsid.pm line 2896
    Ska::Starcheck::Obsid::set_proseco_probs_and_check_P2(Ska::Starcheck::Obsid=HASH(0x55f98ecb88e8)) called at ./starcheck/src/starcheck.pl line 536

On Mac

(ska3) ➜  starcheck git:(commands-v2-flight-minimal) ✗ env KADI_COMMANDS_VERSION=2 ./sandbox_starcheck -dir ~/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls
Using backstop file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/CR225_1108.backstop
Using guide summary file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/mps/mgAUG1322A.sum
Using OR file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/mps/or/AUG1322_A.or
Using maneuver file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/mps/mmAUG1322A.sum
Using DOT file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/mps/mdAUG1322A.dot
Using mech check file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/output/TEST_mechcheck.txt
Using fidsel file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/History/FIDSEL.txt
Using dither file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/History/DITHER.txt
Using radmon file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/History/RADMON.txt
Using simtrans file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/History/SIMTRANS.txt
Using simfocus file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/History/SIMFOCUS.txt
Using attitude file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/History/ATTITUDE.txt
Using characteristics file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/mps/ode/characteristics/CHARACTERIS_18JAN21
Using aimpoint file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/output/AUG1322A_dynamical_offsets.txt
Using config file /Users/aldcroft/git/starcheck/starcheck/data/characteristics.yaml
Using odb file /Users/aldcroft/git/starcheck/starcheck/data/fid_CHARACTERISTICS
Using agasc_file file /Users/aldcroft/ska/data/agasc/proseco_agasc_1p7.h5
Using manerr file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/output/AUG1322A_ManErr.txt
Using processing summary file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/mps/msAUG1322A.sum
Using TLR file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/CR225_1108.tlr
Using banned_agasc file /Users/aldcroft/git/starcheck/starcheck/data/agasc.bad
Using bad_pixel file /Users/aldcroft/git/starcheck/starcheck/data/ACABadPixels
Using acq_star_rdb file /Users/aldcroft/git/starcheck/starcheck/data/bad_acq_stars.rdb
Using gui_star_rdb file /Users/aldcroft/git/starcheck/starcheck/data/bad_gui_stars.rdb
Dither status in kadi commands does not match DITHER history
Read 258 ACA bad pixels from /Users/aldcroft/git/starcheck/starcheck/data/ACABadPixels
Read 46 bad AGASC IDs from /Users/aldcroft/git/starcheck/starcheck/data/agasc.bad
#####################################################################
# calc_ccd_temps run at Mon Aug 29 06:31:55 2022 by aldcroft
# Continuity run_start_time = 2022:241:10:31:55.222
# calc_ccd_temps version = 13.15.2.dev7+ge7c6691
# chandra_models version = 3.40.2
# kadi version = 6.0.1
#####################################################################

Using backstop file /Users/aldcroft/ska/data/mpcrit1/mplogs/2022/AUG1322/ofls/CR225_1108.backstop
Found 2255 backstop commands between 2022:225:11:37:00.000 and 2022:234:01:26:00.000
RLTT = 2022:225:11:37:00.000
sched_stop = 2022:234:01:26:00.000
Fetching telemetry between 2022:224:11:37:00.000 and 2022:225:11:37:00.000
Getting commands from recent only scenario=None
Getting commands from recent only scenario=None
Merging cmds_recent with archive commands from 2022:195:11:21:50.816
Loaded /Users/aldcroft/ska/data/kadi/cmds2.h5 with 1418496 commands
Selecting commands from cmds_arch[0:]
Matching blocks for (a) recent commands and (b) existing HDF5
  Match(a=0, b=0, size=1)
  Match(a=6, b=1, size=13)
  Match(a=20, b=14, size=7556)
  Match(a=7576, b=7570, size=0)
Diffs between (a) recent commands and (b) existing HDF5
  ('equal', 0, 1, 0, 1)
  ('delete', 1, 6, 1, 1)
  ('equal', 6, 19, 1, 14)
  ('delete', 19, 20, 14, 14)
  ('equal', 20, 7576, 14, 7570)
Getting commands from archive + recent scenario=None
Merging cmds_recent with archive commands from 2022:045:11:21:50.816
Getting commands from archive + recent scenario=None
Calculating ACA thermal model
Propagation initial time and ACA: 2022:225:11:21:50.816 -10.37
Making temperature check plots
Writing plot file starcheck/ccd_temperature.png
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int' at line 45
 at ./starcheck/src/starcheck.pl line 37.
        main::__ANON__("TypeError: unsupported operand type(s) for +: 'NoneType' and "...) called at (eval 78) line 3
        Ska::Starcheck::Obsid::proseco_probs("__main__", "proseco_probs", HASH(0x7fab39c76670)) called at starcheck/src/lib/Ska/Starcheck/Obsid.pm line 2896
        Ska::Starcheck::Obsid::set_proseco_probs_and_check_P2(Ska::Starcheck::Obsid=HASH(0x7faaca4640a8)) called at ./starcheck/src/starcheck.pl line 536
jeanconn commented 2 years ago

It seems like it would be fine to print out the kadi scenario around https://github.com/sot/starcheck/blob/69c72ac732acbc609d70799aa6d04720dc1a62a6/starcheck/src/starcheck.pl#L567 (either for the non-default case or just in general)

taldcroft commented 2 years ago

Running starcheck gives this red output unless the scenario is flight:

image
taldcroft commented 2 years ago

just not sure in this case if the default setting needs to be conditional.

I spent about 3 hours today improving on_head_network and I'm sure that in 3 months I'll forget the right way to run starcheck standalone so it uses the commands archive v2 that I spent months working on. (The answer being the non-obvious env KADI_SCENARIO="" starcheck. So please just throw me a bone here.

jeanconn commented 2 years ago

Sure. I just wanted to make sure I understood this and thought this was an area where it could perhaps be simpler. Fine to introduce a little complexity for convenience and a plus overall that the on_head_network should be more robust (instead of dealing with oddities there one machine at a time).

jeanconn commented 2 years ago

I approved and tested this before the printing change. I note, I'm seeing "Kadi scenario: None" on fido, so I'm not sure that the environment variable is actually set for the perl code.

jeanconn commented 2 years ago

I put in another PR for this issue https://github.com/sot/starcheck/pull/394 as getting the env var from the Python side seems to give the correct answer.