search

tachtler / dovecot-backup

Dovecot backup shell script for saving emails for every mailbox to its own tar.gz file.
GNU General Public License v3.0
56 stars 20 forks source link

feat: Improved FreeBSD compatibility #26

Closed wombelix closed 1 year ago

wombelix commented 1 year ago

Logic added to run commands with different parameters and / or arguments if executed on FreeBSD.

Tested with:

Out of the box, dovecot_backup.sh was throwing errors in different stages due to wrong parameters. After this adjustments, I could use it to export multiple mailboxes with around 30GB of mails without issues.

I want to share what worked for me in the spirit of open source, thanks for that helpful tool!

Dom

tachtler commented 1 year ago

Hi Dominik,

I will have a look at the pull request later.

Thanks for your help in improving the script, in advance!

Greetings Klaus.

tachtler commented 1 year ago

Hi Dominik,

I've sent you a version 1.20 of the script dovecot_backup.sh via email. Would you be so kind and try this also under FreeBSD?

Thanks in advance.

Greetings Klaus.

wombelix commented 1 year ago

I've sent you a version 1.20 of the script dovecot_backup.sh via email. Would you be so kind and try this also under FreeBSD?

My pleasure, worked without issues, but I saw that the Runtime calculation in my PR was wrong, I pushed a new version which introduce this change:

@@ -700,8 +740,8 @@
 log ""
 END_TIMESTAMP=`$DATE_COMMAND '+%s'`
 if [ "$OSTYPE" = "FreeBSD" ]; then
-       DIFF_TIMESTAMP=$(($RUN_TIMESTAMP-$END_TIMESTAMP))
-       log "Runtime: `$DATE_COMMAND -r $DIFF_TIMESTAMP +'%H:%M:%S'` time elapsed."
+       DELTA=$((END_TIMESTAMP-RUN_TIMESTAMP))
+       log "$(printf 'Runtime: %02d:%02d:%02d time elapsed.\n' $((DELTA/3600)) $((DELTA%3600/60)) $((DELTA%60)))"
 else
        log "Runtime: `$DATE_COMMAND -u -d "0 $END_TIMESTAMP seconds - $RUN_TIMESTAMP seconds" +'%H:%M:%S'` time elapsed."
 fi

Result of a test run:

root@web[jail:iredmail]:~ # ./dovecot_backup.v120.sh

+-----------------------------------------------------------------+
| Start backup of the mailboxes [Mon, 03 Jul 2023 23:01:37 (+0200)] |
+-----------------------------------------------------------------+

SCRIPT_NAME.................: dovecot_backup

OS_TYPE.....................: FreeBSD

COMPRESSION.................: gz

TMP_FOLDER..................: /var/vmail/backup/dovecot
DIR_BACKUP..................: /var/vmail/backup/dovecot

MAIL_RECIPIENT..............: dominik@wombacher.cc
MAIL_STATUS.................: N

FILE_USERLIST...............: userlist.txt
FILE_USERLIST_VALIDATE_EMAIL: N

Check if command '/usr/local/bin/doveadm' was found .......[ OK ]
Check if command '/usr/bin/tar' was found .................[ OK ]
Check if command '/usr/bin/touch' was found ...............[ OK ]
Check if command '/bin/rm' was found ......................[ OK ]
Check if command '/bin/cat' was found .....................[ OK ]
Check if command '/bin/date' was found ....................[ OK ]
Check if command '/bin/mkdir' was found ...................[ OK ]
Check if command '/usr/sbin/chown' was found ..............[ OK ]
Check if command '/bin/chmod' was found ...................[ OK ]
Check if command '/usr/bin/grep' was found ................[ OK ]
Check if command '/usr/bin/mktemp' was found ..............[ OK ]
Check if command '/bin/mv' was found ......................[ OK ]
Check if command '/usr/bin/stat' was found ................[ OK ]
Check if command '/usr/sbin/sendmail' was found ...........[ OK ]
Check if command '/usr/bin/gzip' was found ................[ OK ]
Check if the script is NOT already runnig .................[ OK ]
Check if TMP_FOLDER exists ................................[ OK ]
Check if TMP_FOLDER owner is vmail ........................[ OK ]
Check if TMP_FOLDER group is vmail ........................[ OK ]
Check if DIR_BACKUP exists ................................[ OK ]
Check if DIR_BACKUP owner is vmail ........................[ OK ]
Check if DIR_BACKUP group is vmail ........................[ OK ]

Check if the variable FILE_USERLIST is set ................[ OK ]
Mailboxes to backup will be read from file.

- File: [userlist.txt]
- Check if FILE_USERLIST exists ...........................[ OK ]
- Check if FILE_USERLIST is readable ......................[ OK ]

+-----------------------------------------------------------------+
| Run backup dovecot_backup ..................................... |
+-----------------------------------------------------------------+

Create temporary '/var/vmail/backup/dovecot/dovecot_backup-x5L6nnh8DuvI' folder [ OK ]

Set ownership of DIR_TEMP to vmail:vmail ..................[ OK ]

Start backup process for user: email@example.com ...
Extract mailbox data for user: email@example.com ...
dsync(email@example.com): Info: Mailbox created: Drafts
dsync(email@example.com): Info: Mailbox created: Junk
dsync(email@example.com): Info: Mailbox created: Sent
dsync(email@example.com): Info: Mailbox created: Trash
dsync(email@example.com): Info: copy from Sent: box=Sent, uid=1, msgid=<c0e3c03936b75f9fe15e3d165e4a19bf@wombacher.it>, size=433, from=team@wombacher.it, subject=Test from email@example.com as team@wombacher.it, flags=(\Seen)
Synchronization done for user: email@example.com ...
Packaging to archive for user: email@example.com ...
a wombacher.it/automation
a wombacher.it/automation/Maildir
a wombacher.it/automation/Maildir/maildirfolder
a wombacher.it/automation/Maildir/tmp
a wombacher.it/automation/Maildir/.Trash
a wombacher.it/automation/Maildir/.Sent
a wombacher.it/automation/Maildir/dovecot.index.log
a wombacher.it/automation/Maildir/.Drafts
a wombacher.it/automation/Maildir/dovecot.list.index.log
a wombacher.it/automation/Maildir/dovecot-acl-list
a wombacher.it/automation/Maildir/new
a wombacher.it/automation/Maildir/.Junk
a wombacher.it/automation/Maildir/dovecot.mailbox.log
a wombacher.it/automation/Maildir/dovecot-uidlist
a wombacher.it/automation/Maildir/subscriptions
a wombacher.it/automation/Maildir/dovecot-uidvalidity
a wombacher.it/automation/Maildir/dovecot-uidvalidity.64a33735
a wombacher.it/automation/Maildir/cur
a wombacher.it/automation/Maildir/.Junk/cur
a wombacher.it/automation/Maildir/.Junk/dovecot-uidlist
a wombacher.it/automation/Maildir/.Junk/new
a wombacher.it/automation/Maildir/.Junk/maildirfolder
a wombacher.it/automation/Maildir/.Junk/dovecot.index.log
a wombacher.it/automation/Maildir/.Junk/tmp
a wombacher.it/automation/Maildir/.Drafts/cur
a wombacher.it/automation/Maildir/.Drafts/dovecot.index.log
a wombacher.it/automation/Maildir/.Drafts/maildirfolder
a wombacher.it/automation/Maildir/.Drafts/new
a wombacher.it/automation/Maildir/.Drafts/tmp
a wombacher.it/automation/Maildir/.Drafts/dovecot-uidlist
a wombacher.it/automation/Maildir/.Sent/tmp
a wombacher.it/automation/Maildir/.Sent/dovecot.index.log
a wombacher.it/automation/Maildir/.Sent/maildirfolder
a wombacher.it/automation/Maildir/.Sent/dovecot.index.cache
a wombacher.it/automation/Maildir/.Sent/dovecot-uidlist
a wombacher.it/automation/Maildir/.Sent/cur
a wombacher.it/automation/Maildir/.Sent/new
a wombacher.it/automation/Maildir/.Sent/cur/1640703637.M290368P61683.web.dominik-wombacher.de,S=433,W=445:2,S
a wombacher.it/automation/Maildir/.Trash/dovecot-uidlist
a wombacher.it/automation/Maildir/.Trash/new
a wombacher.it/automation/Maildir/.Trash/maildirfolder
a wombacher.it/automation/Maildir/.Trash/cur
a wombacher.it/automation/Maildir/.Trash/dovecot.index.log
a wombacher.it/automation/Maildir/.Trash/tmp
Delete mailbox files for user: email@example.com ...
Delete mailbox files at: /var/vmail/backup/dovecot/dovecot_backup-x5L6nnh8DuvI [ OK ]
Copying archive file for user: email@example.com ...
Move archive file for user to: /var/vmail/backup/dovecot ..[ OK ]
Delete archive files for user: email@example.com ...
Delete old archive files from: /var/vmail/backup/dovecot ..[ OK ]
Ended backup process for user: email@example.com ...

Delete temporary '/var/vmail/backup/dovecot/dovecot_backup-x5L6nnh8DuvI' folder [ OK ]

Set ownership of DIR_BACKUP to vmail:vmail ................[ OK ]
Set permission of DIR_BACKUP to drwx------ ................[ OK ]
Set file permissions in DIR_BACKUP to -rw------- ..........[ OK ]

+-----------------------------------------------------------------+
| End backup dovecot_backup ..................................... |
+-----------------------------------------------------------------+

+-----------------------------------------------------------------+
| Runtime statistics ............................................ |
+-----------------------------------------------------------------+

- Number of determined users: 1
- ...Summary of failed users: 0

Runtime: 00:00:01 time elapsed.

+-----------------------------------------------------------------+
| Finished creating the backups [Mon, 03 Jul 2023 23:01:38 (+0200)] |
+-----------------------------------------------------------------+

root@web[jail:iredmail]:~ # ll /var/vmail/backup/dovecot/email@example.com-dovecot_backup_20230703_230137.tar.gz 
-rw-------  1 vmail  vmail  2810 Jul  3 23:01 /var/vmail/backup/dovecot/email@example.com-dovecot_backup_20230703_230137.tar.gz

Should be ready for v1.20 :)

tachtler commented 1 year ago

Hi Dominik,

I will release version 1.20 today, including further customization, so I will close your pull request, but your changes are of course included in version 1.20.

Thanks again for your commitment.

Greetings Klaus.

tachtler commented 1 year ago

Version: 1.20 Description: GitHub: Pull request #26 Improved FreeBSD compatibility. Thanks to wombelix (Dominik Wombacher)