search

mhausenblas / burry.sh

Cloud Native Infrastructure BackUp & RecoveRY
Apache License 2.0
260 stars 38 forks source link

Backup operation is taking too long #25

Open maikcatrinque opened 5 years ago

maikcatrinque commented 5 years ago

Awesome Project! I just have one issue with it... I have an application being orchestrated by Zookeeper and the backups of zookeeper are taking about 3 hours to run completely. There is a way to make it faster? There is a parameter to collect specific znodes from zookeeper?

mhausenblas commented 5 years ago

Thanks and unfortunately, at the current point there is no way to restrict the znodes.

gmenegatti commented 5 years ago

Hi @mhausenblas, thanks for your quick reply.

@maik-catrinque works with me, but he was not detailed and specific as necessary.

We got that there is no way to select a specific ZNode for backup, but is it considered "normal" for a backup using Burry to take 3-4 hours for a ZooKeeper cluster with 5 nodes and a data size of 10GB?

Just to add more context/information, the size of this zipped backup is around 3GB.

Another example: A new ZooKeeper cluster with 3 nodes and almost no data inside takes up to 2-3 minutes to backup. So if almost no data inside it can take 2-3 minutes, it might be a problem if this stores a lot of data.

Please, is there anything we could do to speed up the backup time or understand why it's taking so long?

Thanks in advance.

mhausenblas commented 5 years ago

I see @gmenegatti and as much as I'd like to be of assistance here, unfortunately I don't have the cycles necessary to dedicate to this project. IOW: I'm not actively working on it anymore, but if anyone of your team wants to step up and contribute to address the performance issues you raised, you're more than welcome.

gmenegatti commented 5 years ago

Got it @mhausenblas. Thanks for your support anyway.

Our team is not an expert in Golang. But we will give it a try.