sot / kadi

Chandra commands and events
https://sot.github.io/kadi
BSD 3-Clause "New" or "Revised" License
5 stars 3 forks source link

Add StarAcq event type and infrastructure for emailed alerts #94

Closed taldcroft closed 7 years ago

taldcroft commented 7 years ago

This adds a new kadi event that has the number of acq stars ID'd and which ones, along with the size of one-shots. This is slightly messy and might not end up being merged. However, it will be run in the short term on the side at 2 hour intervals to provide visibility into metrics related to gyro bias variations and mitigation. This might belong elsewhere (like attitude_error_mon), but decided the existing infrastructure in kadi was useful for this.

Note also the new flag to use MAUDE instead of CXC as the telemetry data source.

To do:

@jeanconn - any opinion on the difference between using localhost vs. head.cfa.harvard.edu for the SMTP server?

jeanconn commented 7 years ago

I don't really have an opinion on the sending mail choice. It looks like our original requirement during GAG migration was that local "sendmail" and direct connections to head.cfa.harvard.edu would continue to work. It looks like direct SMTP connections to localhost are doing the right thing too and we've been moving to that in more and more code.

jeanconn commented 7 years ago

For https://github.com/sot/attitude_error_mon/issues/9 I was also planning to add the one shot pitch/yaw breakdown (or a delta quaternion), and I figured it made sense to do it in kadi, so nice to have it done. Where is this running on the side?

taldcroft commented 7 years ago

It's running on /proj/sot/ska/dev-kadi every 2 hours (except not at 2 and 4am, no point).

I'll change alerts to aca_alert, just wanted to make sure it didn't blow up.

taldcroft commented 7 years ago

I purposely set these limits pretty low for increased awareness in the near term. But note that 60 is not that low from a historical perspective since fewer than 70 of 9000 one-shots in the last 5 years exceed that. So seeing 60 means something slightly unusual has happened and if we see it every week that means one shots are worse.

The alert on n_id <= 5 is definitely too jumpy for production.

jeanconn commented 7 years ago

I don't immediately see why this kicked off two different notes for the 19504 acquisition:

One shot at 2017:121:04:14:26.632
Roll  : 1.65
Pitch : -63.57
Yaw   : 27.50

One shot at 2017:121:04:14:25.898
Roll  : 1.65
Pitch : -63.57
Yaw   : 27.50
jeanconn commented 7 years ago

I'm seeing events_staracq in /proj/sot/ska/data/kadi/events.db3, so did this end up in flight or was that a previously-existing thing?

taldcroft commented 7 years ago

Will leave this as a branch but closing since we have no plans to merge to master.