simonsobs / sisock

Sisock ('saɪsɒk): streaming of Simons Obs. data over websockets for quicklook
Other
2 stars 0 forks source link

data feed server websocket ping times out #41

Closed BrianJKoopman closed 5 years ago

BrianJKoopman commented 5 years ago

The data feed server has been seen timing out and dropping its connection, terminating the server.

2019-06-21T22:22:21+0000 Received published data from feed: observatory.LSA22ZC.feeds.temperatures
2019-06-21T22:22:21+0000 dropping connection to peer tcp4:172.20.0.2:8001 with abort=True: WebSocket ping timeout (peer did not respond with pong in time)
2019-06-21T22:22:21+0000 session closed with reason wamp.close.transport_lost [WAMP transport was lost without closing the session 2585746409701808 before]
2019-06-21T22:22:21+0000 disconnected
2019-06-21T22:22:21+0000 Main loop terminated.

This is similar to something that was going on in the ocs Agents a while back: https://github.com/simonsobs/ocs/issues/17

The problem was that data publishing should have been called from the reactor thread. Likely something similar is happening here when we receive the data.