Open markstos opened 10 years ago
@markstos interesting concept. Seems like maybe a --limit option on restartall? this would execute for-each-limit type functionality rather than restart them all at once.
@jcrugzz Something like that sounds right. To me, something like --max-at-once
would be more evocative, but I think I'd use it by any name if it existed.
@markstos I think the best about for this would be as a configuration value and not a CLI option.
e.g.
forever config set rolling true # enables rolling restart
forever config set rolling 5 # enables rolling restart with a concurrency of 5
That would be OK with me too. I can see why a config value might be better-- Someone who's familiar with the difference might set up the 'forever' configuration but later other devs are assigned to "just restart" something and not think about the details-- or may know which method is used.
Thanks for considering this!
Mark
On Thu, Sep 11, 2014 at 4:07 AM, Charlie Robbins notifications@github.com wrote:
@markstos https://github.com/markstos I think the best about for this would be as a configuration value and not a CLI option.
e.g.
forever config set rolling true # enables rolling restart forever config set rolling 5 # enables rolling restart with a concurrency of 5
Reply to this email directly or view it on GitHub https://github.com/nodejitsu/forever/issues/603#issuecomment-55232673.
Mark Stosberg
Senior Systems Engineer
RideAmigos
I have the same situation, my apps are running like a queue, for example:
forever start app1
forever start app2
forever start app3
app1
is the levelup
sever and provides restful apis for app2
& app3
, when I using forever restartall
, all the monitors have been restarted out of order, It seems app2
is running, but app1
is still restarting, the bad news is: app2
crashed.
So, i think maybe all the processes could be sorted by index when you invoke getProcesses
method, and restart one by one.
I'm not sure what happened to the commit above. Here's an alternate pull request which implements just the rolling-restart request:
Could I have some updated feedback on the PR I proposed above. It proposes always using a rolling restart for the "restart" cases with multiple nodes, to avoid downtime when all the nodes have stopped, but none have started yet.
This mirrors the zero downtime upgrade approach that Nginx uses. I'll note that Nginx does not offer configuration options for how a restart happens. The rolling zero-downtime approach is the only option. I recommend emulating Nginx's successful design here.
If you really want to take the entire cluster offline before bring up workers with a new configuration, there is already an option to do this, just like there is in Nginx: Just issue a stop
command followed by a start
command.
I'll note that the StrongLoop slc
tool uses rolling starts. Search for "rolling restart" on this page to see it mentioned.
Also note this chart comparing pm2
with StrongLoop Process Manager
and forever
. There's a line item for "Rolling Restarts" as a desirable feature which the two competing options have, but forever
does not: http://strong-pm.io/compare/
pm2
offers several options:
Implementing the rolling-start as "reload" instead of "restart" as option, but I think it could just be considered an improvement to "restart", without requiring people to figure which of 4 different options for restarting their app is best.
I have several produces managed by
forever
which work as a pool. The reverse proxy in front of them can gracefully handle some members of the pool being offline.However, it appears that when 'forever restartall' is used, all my node instances are going offline at once, and the app temporarily becomes unavailable. This behavior is nice in some ways in that it's fast and keeps all the nodes together in a consistent state.
Still, it would be nice to have an option for a throttled version of restartall that restarts one at a time, perhaps with a pause.
As a workaround, I could write a script to query
forever list
and then callforever restart
on the node instances one by one.