HWP speed is null for observations before 1712000000 even hwp was spinning

[tterasaki]

When you load data of obs_id = 'obs_1710588251_satp3_1111111'

ctx = core.Context('/so/metadata/satp3/contexts/use_this.yaml')
obs_id = 'obs_1710588251_satp3_1111111'
meta = ctx.get_meta(obs_id)
plt.plot(meta.hwp_solution.hwp_angle)

It looks the HWP was spinning in the run.

But when you load the obsdb, the hwp speed is set to be none:

obs_rset = ctx.obsdb.query(query_text="obs_id = 'obs_1710588251_satp3_1111111'",)
obs_rset[0]
>>>
OrderedDict([('obs_id', 'obs_1710588251_satp3_1111111'),
             ('timestamp', 1710588253.2851167),
             ('start_time', 1710588253.2851167),
             ('stop_time', 1710591900.6801167),
             ('n_samples', 729480),
             ('telescope', 'satp3'),
             ('telescope_flavor', 'sat'),
             ('tube_slot', 'st1'),
             ('tube_flavor', 'mf'),
             ('type', 'obs'),
             ('subtype', 'cmb'),
             ('wafer_count', 7),
             ('detector_flavor', 'tes'),
             ('duration', 3647.3949999809265),
             ('az_center', 228.5099587064278),
             ('az_throw', 20.68926185784221),
             ('el_center', 59.99950819241318),
             ('el_throw', 0.0011854834204427789),
             ('roll_center', 0.0039296238691403675),
             ('roll_throw', 0.00015968712118198647),
             ('hwp_freq_mean', None),
             ('hwp_freq_stdev', None)])

So I would like to suggest that manifest db of hwp_angle have column of hwp_freq_mean and hwp_freq_stdev in itself. As I am doing in pwv database (https://github.com/simonsobs/sotodlib/blob/3b6cc16603b08019289b44567ea7afe1fd6ffd83/sotodlib/site_pipeline/make_cosamp_hk.py#L267-L278). And make it queryable with the new query style which I am PR-ing (https://github.com/simonsobs/sotodlib/pull/856/files).

[mhasself]

This particular obs' bad data is some kind of obsdb bug that needs to be resolved (I can tell because the Book M_index.yaml does record the hwp stats).

I think loading more detailed hwp info, at the extended-obsdb level, is a really good idea... however, if/when you do that, please make include the direction of rotation somehow! That's a big thing we'll want to use for grouping and selecting data.

[AERAdler]

Hi, This obs predates the inclusion of hwp_freq in the update_obsdb search (see issue #777, issue #672, and PR #780) I made an up to date version of the obsdbs for satp1 and satp3 on March 28th, located in /so/home/aeadler/updated_obsdb on the site computing. We can either merge it with the current obsdbs or I can run a new update of those files.