UWMRO / TCS

4 stars 6 forks source link

RA & Dec display gets "stuck" #13

Open ojf opened 7 years ago

ojf commented 7 years ago

Not sure what triggers this, but starting an entry to track it. Team reported, "Noticed that the ra and dec froze on our last target. When we slewed, they didn't update from our last coordinates. Everything else in the Bifrost TTC Status area is fine and working properly. The coordinates are the only thing that's stuck."

Confirmed, I've seen it too.

dougbrn commented 7 years ago

This was common during development and is typically associated with the server freezing/ the driver connection to the telescope being interrupted. While to my knowledge sometimes this just happens for no good reason, one cause I was able to hunt down was a memory leak that occurs when the status log gets too many lines in it. In development when the status log was verbose for testing purposes, it would get too large quickly and start to throttle the memory usage for the rest of the GUI, when this happens the GUI loses ability to keep up with the threaded clock cycles for telescope communication in a stable fashion (checking for a paddle every 0.1 seconds for example). I "corrected" for this by reducing the amount of information put into the logs per second, but over long enough observing periods this could add up and reintroduce as an issue if the observers were active enough.

I doubt this would fix it all together, but my suggestion here would be to introduce limits to the amount of lines the log boxes can hold before deleting lines.

dougbrn commented 7 years ago

Also important to note that when the rest of the Status Area is "fine and working properly" this is because these quantities are updated internally and don't rely on communication with the telescope.