search

subutai-io / cdn

Gorjun is a golang replacement for Kurjun project.
Apache License 2.0
19 stars 13 forks source link

in future. clustering gorjun's database #247

Open etursunbaev opened 6 years ago

etursunbaev commented 6 years ago

I think in future it should be implemented database clustering. For now there is one master node and no backups. It is dangerous.

emli commented 6 years ago

@etursunbaev can you write some details and proves? As I know we backup databases in prod CDN. Do we do backups of artifacts?

etursunbaev commented 6 years ago

As you know only one master prod works so imagine in some situations this master will fail than all related systems will fail. From infra side we always ready help to you. Even if backups exist it might take several hours to restore system.

dilshat commented 6 years ago

As a quick solution we can consider Kurjun db and artifacts replication to some standby node that would be turned into master when there is a problem with current master. Of course, a proper monitoring setup is crucial here. Monitoring will make the transition seamless

As a long term solution we should separately consider Kurjun database and Kurjun artifacts: 1) Kurjun artifacts can be only replicated to some standby or active node 2) Kurjun database contains metadata: template and user information, sharing and permissions. It should be based upon a database implementation that supports clustering (afaik BoltDB does not support it) For solving (2) we can either extend BoltDB with additional functionality, e.g. see https://github.com/vincent-petithory/rbolt Or better use another database that supports clusterization out-of-the-box, e.g. see https://github.com/tidwall/summitdb or https://github.com/siddontang/ledisdb

emli commented 6 years ago

@etursunbaev can you report the current state of backups? Do we do backups of database and files?