tlaverdure / laravel-echo-server

Socket.io server for Laravel Echo
MIT License
2.65k stars 510 forks source link

Laravel-echo-server stops working without an error in logs #444

Open DaGLiMiOuX opened 4 years ago

DaGLiMiOuX commented 4 years ago

I've been working for almost 1 year with laravel-echo-server for my laravel project. The project was working perfectly and it was being used for real-time communication between the clients and the server to show real-time server usage and some kind of chats.

I've created a .service file in the server using CentOS where the project is located. After a while working in production, the laravel-echo-server stopped working until I restarted the service. I thought that it could be like a "random problem" in my server, and I ignored it.

This year, the service stopped working 2 times and still no errors in logs (take a look at the screenshot attached below). If you take a look at the picture where the last error ocurred, you can see that the service stopped working on September, 2nd until I restarted it on September, 11th, and these are the logs reported by journalctl.

Now I'm facing that I have to do something with this, because even if it only occurs every 6-8 months, the whole system stops working and it is not something that we can assume in production. I do not know if anyone experienced the same problem or if you are aware of this and have a solution for it. I've tried to Google this problem, but I couldn't find anything related.

This is the server structure:

I do not know if there is a fix for this in a further versions, I'll be waiting for your answers.

Thanks for your time.

b9140051d4437beb2a41e28148ddd6d3

shafter commented 4 years ago

i use supervisor and a cronjob for a daily restart 0 4 * supervisorctl restart laravel-echo

shafter commented 4 years ago

and check your failed_jobs log

DaGLiMiOuX commented 4 years ago

Thanks for the information.

This question may sound stu***, but how should a Laravel failed job have some influence on node socket server like laravel-echo-server? I mean that jobs deliver something through sockets or not (if doesnt pass a check or if an error is thrown). I dont see how could it block the service.

Btw, I was trying to find a solution without restarting the service. I dont know if that is possible.

shafter commented 4 years ago

I can remember i had some problems with huge objects and errors inside the event classes crashed my echo server, also the server is in version 1.5 maybe updating fix your problems.

DaGLiMiOuX commented 4 years ago

Answering to @shafter suggestions of checking errors in jobs, etc. I did not have any errors in my jobs. In fact, there were no failed jobs since February this year and it was because the email service from the domain provider stopped working.

Also, I did not send huge objects. Only useful and required informations (3 to 6 object properties). As you say, I will try to update it in my dev environment and, if nothing breaks up, I'll update on production, but I would suggest that for future versions if there is an error or something that stops the server working, even if it is up, should be notified. For example, if they setup a timeout config that we can modify and if a request takes more than that timeout, throw an exception and we can get some information about when happened, the event name or something like that and keep working for other events instead of getting freezed.

It is just a suggestion...