mpalmer / lvmsync

Synchronise LVM LVs across a network by sending only snapshotted changes
http://theshed.hezmatt.org/lvmsync
GNU General Public License v3.0
380 stars 60 forks source link

Problem with apply changes #41

Open WMP opened 9 years ago

WMP commented 9 years ago

Hello, i again testing lvmsync. In original lvm i have file: /var/log/nginx/redmine_access.log with last from today: 12/Feb/2015. When i execute my script:

#!/bin/bash
usage(){
cat << EOF
Użycie: $0 -s HOST -v VG -o LVM...
Ten skrypt tworzy backup woluemnty LVM na serwerze HOST. Wykorzystuje do tego metodę lvmsync.

OPCJE:
   -s HOST     Domena
   -v VG             Nazwa docelowego Volume Group na serwerze HOST
   -o VG             Nazwa oryginalnego Volume Group na tym serwerze
   -h          Pomoc, ten ekran

EOF                                                                                                                                                                                                       
}                                                                                                                                                                                                         
server=                                                                                                                                                                                                   
virtualgroup_server=                                                                                                                                                                                      
virtualgroup_original=                                                                                                                                                                                    
while getopts “s:v:o:h” option                                                                                                                                                                            
do
      case $option in
          s)
              server=$OPTARG
              ;;
          v)
              virtualgroup_server=$OPTARG
              ;;
          o)
              virtualgroup_original=$OPTARG
              ;;
          ?|h)
              usage
              exit 1
              ;;
      esac
done

shift $(($OPTIND - 1))
if [[ -z $server ]]
then
      echo "Error: Musisz podać wartość dla opcji -s"
      echo
      usage;
      exit 1
fi
if [[ -z $virtualgroup_server ]]
then
      echo "Error: Musisz podać wartość dla opcji -v"
      echo
      usage;
      exit 1
fi
if [[ -z $virtualgroup_original ]]
then
      echo "Error: Musisz podać wartość dla opcji -o"
      echo
      usage;
      exit 1
fi
if [[ $# -lt 1 ]] 
then
      echo "Error: Musisz podać argument którym jest ściezka do wolumenty lvm"
      echo
      usage;
      exit 1
fi

while (( "$#" )); do 
  echo $1:;
  if [[ ! -e /dev/$virtualgroup_original/$1 ]]; then
    echo "local: Plik /dev/$virtualgroup_original/$1 nie istnieje!"
    exit 1;
  fi
  echo -n "$server: "; FileExists=`ssh $server "test -e /dev/$virtualgroup_server/$1 && echo 1 || echo 0"`
  size=$(lvdisplay --units m $virtualgroup_original/$1 | grep "LV Size" | awk '{print $3}' | sed -e 's/\..*//')
  snapshot_size=$(($size/10))
  lvcreate --snapshot -L "$snapshot_size"m -n $1-snapshot $virtualgroup_original/$1
  if [[ ${FileExists} -eq 0 ]]; then
    echo -n "$server: "; ssh $server "lvcreate -L "$size"m -n $1 $virtualgroup_server"
    dd if=/dev/$virtualgroup_original/$1-snapshot bs=1M | pv -c | ssh $server dd of=/dev/$virtualgroup_server/$1 bs=1M
  fi
  echo 'lvmsync...'
  echo -n "$server: "; ssh $server "mkdir /snapback/$1 > /dev/null 2>&1"
  lvmsync -v /dev/$virtualgroup_original/$1-snapshot $server:/dev/$virtualgroup_server/$1 --snapback /snapback/$1/$(date +%Y-%m-%d-%H:%M)
  echo 'lvmsync done'
  echo 'cleanup'
  lvremove -f /dev/$virtualgroup_original/$1-snapshot
  shift 
done

And this is output:

root@gaja ~ # ./lvmbackup.sh -s backup2 -v vg0 -o virtual redmine2-disk
redmine2-disk:
backup2:   Logical volume "redmine2-disk-snapshot" created
lvmsync...
backup2: Data source: /dev/mapper/virtual-redmine2--disk
Sending chunk 0..4095...
Seeking to 0 in /dev/mapper/virtual-redmine2--disk
Sending chunk 10875834368..10875838463...
Seeking to 10875834368 in /dev/mapper/virtual-redmine2--disk
Sending chunk 10938511360..10938515455...
Seeking to 10938511360 in /dev/mapper/virtual-redmine2--disk
Sending chunk 10938515456..10938519551...
Seeking to 10938515456 in /dev/mapper/virtual-redmine2--disk
Sending chunk 10938519552..10938523647...
Seeking to 10938519552 in /dev/mapper/virtual-redmine2--disk
Transferred 20480 bytes in 0.23 seconds
You transferred your changes 1048576.00x faster than a full dd!
lvmsync done  
cleanup
  Logical volume "redmine2-disk-snapshot" successfully removed
root@gaja ~ #

But when after this i mount in backup2 redmine2-disk: mount /dev/vg0/redmine2-disk /mnt/ in file /mnt/var/log/nginx/redmine_access.log i have last entry from 06/Feb/2015, so from dd clone.

mpalmer commented 9 years ago

You'll need to work out what block(s) being written that aren't being picked up by lvmsync. The easiest way to do that will be to capture a blktrace output of completed writes, with a command something like this:

blktrace -d /dev/vg0/backup2 -a write -o - | blkparse -i - | grep ' C '

That'll give you a bunch of lines that describe offset and length pairs, in units of sectors. If you can compare that list against the list of blocks that lvmsync copies, that'll be useful. Assuming that actually shows something that lvmsync has missed, we get to dig into why it was missed, but let's take it one step at a time.