akhenry / openmct-yamcs

Open MCT YAMCS plugin
14 stars 9 forks source link

Serialized messages that exceed Yamcs' write buffer length cause WebSocket errors #397

Closed akhenry closed 1 year ago

akhenry commented 1 year ago

Summary

Impacts both Open MCT and Yamcs. Initially we need a minimal reproduction in Yamcs-Quickstart using the default high water mark there.

Yamcs will drop a socket if the buffer fills up and is no longer writable. This appears to be due to a potential impedance mismatch between the buffer used to store the serialized message, and the buffer used to write out to the WebSocket.

The prevailing theory is that this occurs when a serialized message is larger than the high water mark of the write buffer.

Issue is being recorded here first so that we can provide a minimal reproduction of the issue using Yamcs Quickstart with the default settings for high and low water marks -

  1. Configure some XTCE for an arbitrarily large binary array that is slightly larger than the default buffer high water mark,
  2. Write a script (in python, node, whatever works) to push some binary data into that parameter, using setParameterValue.
  3. Navigate to the parameter in the Yamcs Web UI and observe that the parameter cannot be viewed.
scottbell commented 1 year ago

@akhenry I think I'm reproducing the issue and I've got a PR with a script and an example XTCE. Here's a sample parameter that I've fed ~2.6MB worth of image data to with the default writeBufferWaterMark:

small-watermark

Note the value is missing, and attempting to go to Historical Data either causes Chrome to "Ah, Snap" or just shows blank values.

Here's with writeBufferWaterMark set to 160000000:

big-watermark

Note the value is visible, and you can go to Historical Data, wait, and see values.