search

sund / auto-gitlab-backup

A simple script to backup your Gitlab data. This script will copy the backup archives of your gitlab installation via rsync, or scp. Also, you can copy backups to Backblaze’s B2 Cloud Storage service.
http://sund.github.io/auto-gitlab-backup/
GNU General Public License v2.0
315 stars 73 forks source link

Config backup names are not kept #49

Closed VizuaaLOG closed 7 years ago

VizuaaLOG commented 7 years ago

Really liking this script, saved me sooo much time trying to figure out bash and rsync. A++++ :)

Only little issue I seem to have is in regards to the config backups. I have configured the script to also automate and move backing up the gitlab configuration, this works brilliantly. I followed the default way of having a configBackups directory within the main Gitlab backups folder. The backups work and are copied over to my other machine via rsync. However, the names of these backups are not kept. The names of the main rake backups are kept though. I thought maybe it was because there was a space in the file names, but this wasn't the case.

For example a backup created today would have the name of gitlabConf-17-05-03 21:27:30. The file on the Gitlab server is correctly named. However, after the transfer on the second server it is instead called GR05BW~T.TGZ

I know very little about rsync so is this some kind of configuration that needs to be done with this? Are you able to reproduce this? Seems strange considering the main backups are working.

sund commented 7 years ago

If I understand you correctly, you modified the script?

If not, the munged filename may be a temp file created by rsync.

VizuaaLOG commented 7 years ago

To start with it was the original script. After seeing this happen I looked through to see if maybe the names are not carried over or if the config backups are not named at all. When doing this I noticed that the date string given to the config backups contains a space and so thought maybe this was causing weird things to happen during the transfer. But turns out this was not the case.

On the server I can still open the garbled file inside 7zip and it is the backup create via your script, but is named incorrectly. Not having any experience with rsync I was just wondering if maybe it's a configuration I need to change.

sund commented 7 years ago

I don't seem to have this problem; the server's config backups /var/opt/gitlab/backups/configBackups and the server I back them up to have the same directory structure:

root@gitlab /var/opt/gitlab/backups> tree
.
├── 1494156614_2017_05_07_gitlab_backup.tar
├── 1494243016_2017_05_08_gitlab_backup.tar
└── configBackups
    ├── gitlabConf-2017-05-04\ 07:05:01.tgz
    ├── gitlabConf-2017-05-05\ 07:05:01.tgz
    ├── gitlabConf-2017-05-06\ 07:05:01.tgz
    ├── gitlabConf-2017-05-07\ 07:05:01.tgz
    └── gitlabConf-2017-05-08\ 07:05:01.tgz

1 directory, 7 files

Perhaps the output from the script will be more informative?

VizuaaLOG commented 7 years ago

Oh! I was just going through all of the outputs to try and provide more information on this. And when SSHed into the storage server the filenames are intact. However, when browsing the same folder via a mapped network drive on Windows it shows incorrectly?

Maybe this is some encoding issue between Ubuntu and Windows? This is very weird.

VizuaaLOG commented 7 years ago

Aha! I found the solution. Turns out it was Samba mangling the file names. I assume converting the encoding to try and be compatible with older operating systems?

If anyone else is also seeing this issue, I will leave the link here for reference. https://superuser.com/questions/458995/files-folders-get-weird-names-and-become-inaccessible-on-samba-share

To basically summarise what is mentioned. Inside your global smb.conf config add mangled names = no and restart samba. You should see the filenames resolve them selves straight away!

@sund Thank you for your help :) Turns out it was related to Samba :(.