Ladder99 / fanuc-driver

Configurable Fanuc Focas data collector and post processor.
Apache License 2.0
74 stars 25 forks source link

MQTT Exception seems to hang Windows Service - 0.6.0 #87

Closed MRIIOT closed 1 year ago

MRIIOT commented 1 year ago

A bug has been identified by comparing logs across six computers within the same environment where fanuc-driver is running as a Windows service. It was observed that on four instances the last log entry suggests that the BackgroundService has been stopped, with no indication of an unhandled exception. However, both fanuc.exe and fanuc-driver Windows service were still running. The timestamp on the last log entry was within several seconds across all four instances. The other two instances indicate a trapped MQTT exception, where the executable has recovered successfully several seconds later. The timestamp on those two log entries was within a few seconds of the other four instances, which have appeared to gracefully come to a halt.

ref: https://learn.microsoft.com/en-us/dotnet/core/compatibility/core-libraries/6.0/hosting-exception-handling

MRIIOT commented 1 year ago

Based on evidence that MQTT connection was broken during this time, it is possible that the following happened.

https://github.com/Ladder99/base-driver/blob/d4fe842eb03ee98bdea3135cc820f3702994d4cb/base/Strategy.cs#L43 is unprotected and eventually calls into https://github.com/Ladder99/fanuc-driver/blob/9e55095bdacee7eb3129dcc5a0d078d6e3ceaf4d/fanuc/transports/MQTT.cs#L97-L143. It is likely that an unhandled exception could occur if the broker connection is broken immediately prior to calling https://github.com/Ladder99/fanuc-driver/blob/9e55095bdacee7eb3129dcc5a0d078d6e3ceaf4d/fanuc/transports/MQTT.cs#L141.

MRIIOT commented 1 year ago

fix 0963892a67b08c0b05cb09172a52a0381301609a