search

ponzu-cms / ponzu

Headless CMS with automatic JSON API. Featuring auto-HTTPS from Let's Encrypt, HTTP/2 Server Push, and flexible server framework written in Go.
https://docs.ponzu-cms.org
BSD 3-Clause "New" or "Revised" License
5.67k stars 387 forks source link

Multiple Ponzu instances #320

Closed prumor closed 4 years ago

prumor commented 4 years ago

Amazing project, really well thought out.

My question is, how can I have multiple instances of ponzu running, with the same data in both? Since the database is embedded, how could this be archived without replacing the database?

Cheers.

olliephillips commented 4 years ago

Hello @prumor! BoltDB which Ponzu uses for its datastore allows one connection at a time. Attempts at further connections will generally block until the first datastore connection is closed. But, as we're dealing with simple files (system.db and analytics.db), bash copy routines or tools like rsync could give your additional ponzu instances "replicas" of the data. If you maintained one instance for write and the rest just performed read ops I guess this would work, but there'd be no consensus, you couldn't expect to be using all instances the same as if it were distributed, for example.

You should outline what you are trying to achieve with multiple Ponzu instances and the community may be able to guide you further.

prumor commented 4 years ago

Thank you @olliephillips for your response.

I am trying to find out, how could ponzu work, in a high load environment, where you probably need to distribute the load between multiple containers/VM's and maintain data consistency across all of them. In other words a cluster.

Thank you!

olliephillips commented 4 years ago

Yes, that's how I'd approach it.

With Docker say, have a single container handling write endpoints, several containers offering read endpoints with a load balancer/reverse proxy routing GET and POST requests to the appropriate containers. Have your read instances rsyncing from your write master at regular intervals.

Note I haven't done this I'm just spitballing :)

This issue, though quite old, first broached the idea of a more distributed data backend.

https://github.com/ponzu-cms/ponzu/issues/202

olliephillips commented 4 years ago

Going to close this one. Please feel free to re-open if need to.

nilslice commented 4 years ago

@prumor - I'd simplify the design as much as possible. If can have a single "authoring" instance (your main Ponzu server) as an origin server and put up horizontally scalable cache boxes, e.g. Varnish, you can handle whatever traffic you need. This is assuming the majority is Read-only traffic of course.