Error while restoring

[ymarinov]

Hello,

I'm using currently the rewrite_proposal_2.0 of Zmbkpose. I'm replication our main mail server, and I'm having an issue with it. Please find the exact log below, and if you have anything to suggest will be glad to hear from you. Have in mind that no one killed the session - it's saying killed ...

2013-03-05 12:32:50 DEBUG: alex@xxx.net - Restauring from 03/03/2013 05:14:03 FULL /usr/local/bin/zmbkpose: line 333: 2459 Killed $CURL_cmd $CURL_args --data-binary "@$backup_file" -u $ADMINUSER:$ADMINPASS "https://$mailhost:7071/home/$mailbox/?fmt=tgz&resolve=skip"

[liandros]

2013/3/5 ymarinov notifications@github.com

Hello,

I'm using currently the rewrite_proposal_2.0 of Zmbkpose. I'm replication our main mail server, and I'm having an issue with it. Please find the exact log below, and if you have anything to suggest will be glad to hear from you. Have in mind that no one killed the session - it's saying killed ...

2013-03-05 12:32:50 DEBUG: alex@xxx.net - Restauring from 03/03/2013 05:14:03 FULL /usr/local/bin/zmbkpose: line 333: 2459 Killed $CURL_cmd $CURL_args --data-binary "@$backup_file" -u $ADMINUSER:$ADMINPASS "https:// $mailhost:7071/home/$mailbox/?fmt=tgz&resolve=skip"

only by the killed word, I suspect that you are reaching your machine memory limit . curl creates the request in memory before to send it to web server. This consumes much memory as the size of the file to send (plus a 15% from what I've seen). If it is true,you must have a large size backup file. It is posible ? You can list the backup files with : $ zmbkpose -l -a alex@xxx.net

[ymarinov]

Well, this is not the issue, i'm sure, because i've changed any limits on this virtual server where i want to replicate our working Zimbra to maximum, and there are bigger mailboxes than alex's one, which pass without any problems.

However, i'll try to import this user later, since now i'm running full restore. I'm very pleased from this package, but (it's not a problem) it's kinda slow, i mean about 13-15GB of mailbox is transferred aprox for about 7 hours. I know that i can start for example 3 instances in order to speed up the process, but is there anything that may be tuned for faster restoring?

I have one more thing that i might need suggestion. Currently i want to keep 2 full backups, and all of the incrementals (every day) between them. As far as i understand the idea behind the Zmbkpose, i need to run the tool for 2 weeks, and at the end of 3rd (when the 3rd full backup is created) i can delete everything older than 2 weeks, and i'll have 2 weeks behind backup. Am I correct?

[liandros]

2013/3/7 ymarinov notifications@github.com

Well, this is not the issue, i'm sure, because i've changed any limits on this virtual server where i want to replicate our working Zimbra to maximum, and there are bigger mailboxes than alex's one, which pass without any problems.

ok.. but if the process was killed by kernel (and not by an another user using kill), then the process is a extreme resource hog . I only can imagine a memory problem in a specific moment when the resource is insufficient .

However, i'll try to import this user later, since now i'm running full restore. I'm very pleased from this package, but (it's not a problem) it's kinda slow, i mean about 13-15GB of mailbox is transferred aprox for about 7 hours. I know that i can start for example 3 instances in order to speed up the process, but is there anything that may be tuned for faster restoring?

Ouch.. it is really slow. zmbkpose is very simple. Zimbra has the core. For backup do a http query ( by time) for items in mailbox with tar.gz as output format. For restore, do a http POST sending the tar.gz file. Zimbra Incorporates these data to mysql database, and filesystem mailbox. I would like to modify zmbkpose, to backup the mailbox into small pieces, giving possibility to parallelism also for restoration. Additionally, to be small operations, it is possible to control memory usage. I need to progress on this.

I have one more thing that i might need suggestion. Currently i want to keep 2 full backups, and all of the incrementals (every day) between them. As far as i understand the idea behind the Zmbkpose, i need to run the tool for 2 weeks, and at the end of 3rd (when the 3rd full backup is created) i can delete everything older than 2 weeks, and i'll have 2 weeks behind backup. Am I correct?

Yes, you are correct. One comment: You want to keep two weeks of backup cycles. A cycle is a full backup, and subsequent incremental. You can only delete old cycles, maintaining integrity in backups. Watch the issue 31 for aditional comments for deleting old backups please.

[ymarinov]

Thank you for the clarifications regarding the backups cycles. Nevertheless, I think you you underestimate the power of your tool for the opensource community! Combined with ldap backup of the config and users, it's perfect solution (so far) for HOT backing up. Have in mind that many companies are just like the one i work for - they don't want to stop AT ALL their mail servers for backup, because of the losts. So, I think you should be very pleased of your code. Also Zimbra is "hungry" for resources for sure since it's based on java, which leads us for "big" hardware.

That's it ;) If i have any other issues, or probably suggestions, will write you down!

Cheers!

[liandros]

2013/3/7 ymarinov notifications@github.com

Thank you for the clarifications regarding the backups cycles. Nevertheless, I think you you underestimate the power of your tool for the opensource community! Combined with ldap backup of the config and users, it's perfect solution (so far) for HOT backing up. Have in mind that many companies are just like the one i work for - they don't want to stop AT ALL their mail servers for backup, because of the losts. So, I think you should be very pleased of your code. Also Zimbra is "hungry" for resources for sure since it's based on java, which leads us for "big" hardware.

Thanks for your comment ymarinov. It is energy to keep working. Thanks.

[ymarinov]

hello again.

just want to discuss something ... is there any chance Zmbkpose to skip the Distribution Lists, or to add similar option to "-u On failure of a backup account, does not stop" for the restore action. I'm asking this, because once i start to restore some accounts at some point something like this breaks the restore process:

2013-03-19 14:41:48 ERROR: file mailbox.tgz does not exists in tar file "/opt/zimbra/backup/spam.3mrsvaake@xxxx.net/20130317030340:INC.tar"

Thank you for your time.

[iristran]

Please help me !! , i have problem with script Hot backup " zmbkpose "for zimbra here my problem: [root@mail ~]# su - zimbra [zimbra@mail ~]$ zmbkpose -f kiettt@xxxxx.com /usr/local/bin/zmbkpose: line 420: /mnt/BACKUP/MailBox/sessions.txt: Permission denied mktemp: failed to create directory via template /mnt/BACKUP/MailBox/XXXX': Permission denied /usr/local/bin/zmbkpose: line 432: /kiettt@xxxxx.com.ldiff: Permission denied grep: /kiettt@xxxxx.com.ldiff: No such file or directory /usr/local/bin/zmbkpose: line 434: /kiettt@xxxxx.com.tgz: Permission denied /usr/local/bin/zmbkpose: line 435: /kiettt@xxxxx.com.Junk.tgz: Permission denied /usr/local/bin/zmbkpose: line 436: /kiettt@xxxxx.com.Trash.tgz: Permission denied /usr/local/bin/zmbkpose: line 437: /mnt/BACKUP/MailBox/sessions.txt: Permission denied mv: cannot stat': No such file or directory /usr/local/bin/zmbkpose: line 451: /mnt/BACKUP/MailBox/sessions.txt: Permission denied du: cannot access /mnt/BACKUP/MailBox/full-20170314090113/*': No such file or directory du: cannot access/mnt/BACKUP/MailBox/full-20170314090113': No such file or directory I map one network drive from windows on /mnt/BACKUP

Please help me, Thank a lot!!!!