WyriHaximus
commented
3 years ago I am listening to hundreds of streams at a 100ms interval and it looks like ratchet starts to swallow and lose messages (or return them in wrong order) when the processing is temporarily lagging behind by 100-200ms.
What does processing entail? Does it involve a database, and if so, how are you connecting to it?
It's difficult to debug due to the gigabytes of data rolling through but I'm on this for about a week now and most likely the problem is Ratchet (+React).
Honestly, I doubt that, but let's see what we can find out about these issues you are facing.
The lag comes when a spike in traffic happens, it's not a problem IF the messages are buffered (I expected that).
Logical, more traffic, means your process has more to do. Ratchet will only buffer until it can parse a full message. So that shouldn't be the issue.
I've tried various React loop implementations (the quickest was UV but it is unstable and crashes after a day), they all seem to have the same issue.
How is it unstable? How did it crash? Did it run out of memory, or did you get an error message?
Is there a buffer limit ?
Whatever you set as memory limit. ReactPHP and thus Rachet will pass you new messages as soon as we receive them.
Is there a way to ensure the messages are processed in order (not sure if React guarantees that)?
ReactPHP guarantees you're getting your data as we get it from the network stack. So that is in order per connection.
Is there a way to check if messages got lost and can we avoid that ?
Update: The order of messages is wrong it seems. I started tracking and it looks like ratchet does not guarantee that the queue of messages is played in the order it's received.
What queue? Inside your app? Or somewhere else? Ratchet will just pass the messages onto you when it gets them
I am listening to hundreds of streams at a 100ms interval and it looks like ratchet starts to swallow and lose messages (or return them in wrong order) when the processing is temporarily lagging behind by 100-200ms. It's difficult to debug due to the gigabytes of data rolling through but I'm on this for about a week now and most likely the problem is Ratchet (+React). The lag comes when a spike in traffic happens, it's not a problem IF the messages are buffered (I expected that).
I've tried various React loop implementations (the quickest was UV but it is unstable and crashes after a day), they all seem to have the same issue.
Is there a buffer limit ? Is there a way to ensure the messages are processed in order (not sure if React guarantees that)? Is there a way to check if messages got lost and can we avoid that ?
Update: The order of messages is wrong it seems. I started tracking and it looks like ratchet does not guarantee that the queue of messages is played in the order it's received.