croservices / cro-core

The heart of the Cro library for building distributed systems in Raku, including pipeline composition and TCP support.
https://cro.services/
Artistic License 2.0
27 stars 9 forks source link

zero downtime update of cro web app #31

Open melezhik opened 3 years ago

melezhik commented 3 years ago

Hi guys! First of all thanks for a great product you do.

I use cro for web application (https://mybf.io), my .cro.yaml file is that:

cro: 1
id: mbf
name: My Butterflies Web
entrypoint: app.raku
ignore:
  - .cache
  - .tom
  - articles
  - conf
  - cro.log
  - js

when I update any file not listed in ignore list it take cro awhile to restart an application to pick up changes, during this time my application is not available and my nginx server returns 502 error.

Any cure for that?

My web app command is:

nohub cro run > cro.log &

Snippet of app.raku running a cro web server is:

my Cro::Service $service = Cro::HTTP::Server.new:
    :host<0.0.0.0>, :port<2000>, :$application;

$service.start;

react whenever signal(SIGINT) {
    $service.stop;
    exit;
}

PS I know it could be hard to fix that on cro side, I am just interested in how someone would solve that

jnthn commented 3 years ago

We intended cro run more as a development convenience than an ideal way to run in production (the cro command line tool is documented as a development tool). Of course, that won't stop anyone... :-)

What I personally do for zero downtime updates is leave the lifting to Kubernetes; essentially, upon a deploy it starts the new container while leaving the old one running, then when the new one is ready it starts to route traffic over to it. (While a readiness probe is the reliable way to do this, I've found that there's already some built-in default delay, and it tends to be long enough for smaller Cro applications anyway.)

melezhik commented 3 years ago

Hi Jonathan ! Thanks for a quick response. I get that, but I am looking for less expressive solution, not like kubernetes )))

Let me maybe reshape my question - can I set up a custom ( aka maintenance ) page using cro to notify users that my app is under update right now. I know nginx could do that, but would be nice if I can do this using Raku/cro ...

Thanks

jnthn commented 3 years ago

I am looking for less expressive solution, not like kubernetes

Yeah, for some things it's like using a flame thrower to swat a fly... :-)

So far as options:

melezhik commented 3 years ago

Yeah, this cro based customized proxy server would be a good idea, at least for small/medium size projects where performance is not that important ...

niner commented 2 years ago

We have found haproxy to be very useful for this. It's small, stable and scalable. With its retry feature, you should be able to hold requests until the backend is ready again after a restart (users would just notice a delay but will get served eventually). If you scale up to multiple backends, you just need to add them to your configuration and get failover that way (i.e. you restart one of the backends and the other will get all requests until the first one is ready again). It also supports "backup" servers out of the box, i.e. backends that will only get used if all primaries are down. Such a backup server can be a simple nginx serving a static maintenance page.

Much of this can be achieved with nginx as frontend proxy as well, but it's much harder to get right and they are pushing their commercial nginx plus offering for this use case hard.