ausocean / cloud

GNU General Public License v3.0
1 stars 1 forks source link

Improve permanent start/transition error behaviour #173

Closed saxon-milton closed 3 days ago

saxon-milton commented 4 days ago

Resolves #156

We're making the error handling behaviour a little smarter when it comes to error condition behaviour. For many of these cases, we were transitioning to the idle state, but this isn't great for a permanent broadcast.

Firstly, if we time out transitioning from live to slate, we're now notifying ops to check fordwarder software and staying in the live state.

Secondly, if we time out on permanent starting, we use the onFailure closure which gives us some more chances, but will notify and disable when a limit is reached.

Thirdly, if we time out transitioning from slate to live, we enter the permanent failure state which stays in slate mode and notifies ops that there may be a problem with the hardware.