Posmoves recent rehome

[julienguy]

• Add option --recent-rehome to declare in get_posmoves which calibration exposure(s) had a recent rehome. This sets to true the values in the column RECENT_REHOME for the moves of these exposures. Ex.

  # use expid=2107 on 2020/06/05, but assume no recent-rehome
  # good RC on 2020/06/08 for most positioners, expid=2119
  get_posmoves --host xxx--port xxx --password xxx --exposure-ids 2107,2119 --petal-id 1 -o . --recent-rehome 2119

• Use this information in the fit: Add as many extra (nuisance) parameters EPSILON_T,P_xxx as there are sequences of "consistent POS_T,P" for which RECENT_REHOME=False. The EPSILON_T,P absorb the drifts of POS_T,P such that the parameters OFFSET_T,P only encode the actual orientation of the positioners in the petal.

• The fitter is now complex and really needs to be reorganized, but it would be useful to commit and tag the current version that is working, because it includes what we want (covariances, outlier rejection, flags and detection of failures, fit of OFFSET_T,P only on rehomed data).

Full example

# use expid=2107 on 2020/06/05, but assume no recent-rehome
# good RC on 2020/06/08 for most positioners, expid=2119
get_posmoves --host beyonce.lbl.gov --port 5432 --password reader --exposure-ids 2107,2119 --petal-id 1 -o . --recent-rehome 2119
# new RC on 2020/06/09 for some positioners, expid=2141, overwrite them
get_posmoves --host beyonce.lbl.gov --port 5432 --password reader --exposure-ids 2141 --petal-id 1 -o . --recent-rehome 2141 --pos-ids M06248,M06223,M06226,M06252,M08138
# new RC on 2020/06/10 for some positioners, expid=2154, overwrite them
get_posmoves --host beyonce.lbl.gov --port 5432 --password reader --exposure-ids 2154 --petal-id 1 -o . --recent-rehome 2154 --pos-ids M08052,M06998,M01985

fit_posparams -i M?????.csv -o .
merge_posparams -i M*_paramfits.csv -o all_paramfits.csv
../../scripts/plot_paramfits.py all_paramfits.csv

GOOD (10 worst):
M06478 rms=0.050 x=69.6 y=23.3
M06520 rms=0.025 x=32.8 y=14.2
M02202 rms=0.024 x=59.1 y= 5.2
M02491 rms=0.023 x=48.6 y= 5.1
M01985 rms=0.021 x=205.4 y=131.8
M06523 rms=0.020 x=43.3 y=14.2
M06998 rms=0.020 x=194.9 y=113.8
M06257 rms=0.020 x=48.6 y=23.3
M08052 rms=0.020 x=148.1 y=14.3
M02182 rms=0.020 x=101.1 y= 5.2

BAD:
FLAGGED:
M01639 FLAGS=8 ['INVALID_TABLE']
M06190 FLAGS=5 ['THETA_STUCK', 'FAILED_FIT']
M06469 FLAGS=8 ['INVALID_TABLE']
M06805 FLAGS=16 ['INVALID_AFTER_FILTER']
M06807 FLAGS=8 ['INVALID_TABLE']
M06923 FLAGS=8 ['INVALID_TABLE']
M06925 FLAGS=16 ['INVALID_AFTER_FILTER']
M06926 FLAGS=16 ['INVALID_AFTER_FILTER']
M06928 FLAGS=16 ['INVALID_AFTER_FILTER']
M07112 FLAGS=5 ['THETA_STUCK', 'FAILED_FIT']
M07699 FLAGS=5 ['THETA_STUCK', 'FAILED_FIT']
M08203 FLAGS=16 ['INVALID_AFTER_FILTER']
M08272 FLAGS=16 ['INVALID_AFTER_FILTER']
M08273 FLAGS=16 ['INVALID_AFTER_FILTER']
M08348 FLAGS=5 ['THETA_STUCK', 'FAILED_FIT']

[joesilber]

This looks good to me. To capture for the future, my current thoughts on next steps are:

• We detect RECENT_REHOME condition automatically by inspecting the logs.
• We consider refactoring the EPSILON functionality.
  • Add an additional pass in fithandler.py, while simplifying fitter.py.
  • We might want to change the passes from STATIC, DYNAMIC to:
    • HARDSTOPS ... uses only a single RECENT_REHOME sequence. Key output is OFFSET_T, P. "Ignore" everything else (though of course still save everything to output file)
      • LENGTHS ... uses all data. Key output is LENGTH_R1, R2 and OFFSET_X, Y
      • SCALES ... uses all data. Key output is SCALE_T and SCALE_P
  • The output file could be changed to have a field like:
    • FIT_TARGETS, string, comma separated list from among LENGTH_R1, LENGTH_R2, OFFSET_X, OFFSET_Y, OFFSET_T, OFFSET_P, SCALE_T, SCALE_P.
    • This indicates which of the calibration parameters we expect to have been well-constrained, given the particular data and options that were fed to the fitter.
    • Or this information could be encoded in FLAGS