search

peter-mount / opendata

Latest & first public version of my code that monitors the UK Rail Network in real time using Open Data feeds
Other
12 stars 2 forks source link

NRE push port feed suddenly stops, inactivity timer doesn't reset the connection #16

Closed peter-mount closed 9 years ago

peter-mount commented 9 years ago

Spotted this when looking at the logs this morning as nothing appeared to be going through Rabbit. It looked like it just stopped at around 11:59 yesterday morning.

I'm thinking its a deadlock in the new JAXB code as thats the only real difference between NRE & Network Rail which has been stable for months.

peter-mount commented 9 years ago

I've changed the locking used by JAXBSupport so it shouldn't be able to deadlock itself anymore

peter-mount commented 9 years ago

Refactored the application so it now just pushes the messages as-is (but uncompressed) to RabbitMQ.

It also has a watchdog which will restart the application if no messages after 3 minutes, emailing the log as necessary.

Looking at the history, even overnight the message rate is such that we should still get messages.

peter-mount commented 9 years ago

Fixed this with a custom inactivity timer which then terminates the app. The app then restarts.

I notice that it does this 2 but usually 3 times just after 0200 daily now, but it's been running for over a week now without reoccuring.

A hack but one that works well.