search

lamw / ghettoVCB

ghettoVCB
MIT License
1.28k stars 364 forks source link

issue in 2019_01_06 release with combinantion of options -e /path/to/file and -d dryrun #182

Open bisi-sysadmin opened 4 years ago

bisi-sysadmin commented 4 years ago

I recently updated a machine to ESXi 6.7u3, from something like 5.5. (actually a completely new install). I also updated the existing ghettoVCB code to the 2019-01-06 release. In doing some testing, I have discovered that if I have the -e option pointing at a file with VMs to be excluded, and add the -d dryrun option, ghettoVCB incorrectly fails to detect the debug level as dryrun, and reports an error. ( Final status: ERROR: No VMs backed up!).

I should note that the -e option does seem to work, in that the machines I want to avoid backing up are not selected and reported in the backup.

Here is the command and the output with all but one VM elided...

[root@esxi0:~] /vmfs/volumes/backupstore0/ghettoVCB/ghettoVCB.sh -a -g /vmfs/volumes/backupstore0/ghettoVCB/ghettoVCB.conf -e /vmfs/volumes/backupstore0/ghettoVCB/exclusion_list -d dryrun
Logging output to "/tmp/ghettoVCB-2020-02-22_03-50-11-2210653.log" ...
2020-02-22 03:50:11 -- info: ============================== ghettoVCB LOG START ==============================

2020-02-22 03:50:11 -- info: CONFIG - USING GLOBAL GHETTOVCB CONFIGURATION FILE = /vmfs/volumes/backupstore0/ghettoVCB/ghettoVCB.conf
2020-02-22 03:50:11 -- info: CONFIG - VERSION = 2019_01_06_4
2020-02-22 03:50:11 -- info: CONFIG - GHETTOVCB_PID = 2210653
2020-02-22 03:50:11 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/backupstore0/_ghettobackups
2020-02-22 03:50:11 -- info: CONFIG - ENABLE_NON_PERSISTENT_NFS = 0
2020-02-22 03:50:11 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 3
2020-02-22 03:50:11 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2020-02-22_03-50-11
2020-02-22 03:50:11 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
2020-02-22 03:50:11 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0
2020-02-22 03:50:11 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0
2020-02-22 03:50:11 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 3
2020-02-22 03:50:11 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5
2020-02-22 03:50:11 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15
2020-02-22 03:50:11 -- info: CONFIG - LOG_LEVEL = dryrun
2020-02-22 03:50:11 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCB-2020-02-22_03-50-11-2210653.log
2020-02-22 03:50:11 -- info: CONFIG - ENABLE_COMPRESSION = 0
2020-02-22 03:50:11 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0
2020-02-22 03:50:11 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0
2020-02-22 03:50:11 -- info: CONFIG - ALLOW_VMS_WITH_SNAPSHOTS_TO_BE_BACKEDUP = 0
2020-02-22 03:50:11 -- info: CONFIG - VMDK_FILES_TO_BACKUP = all
2020-02-22 03:50:11 -- info: CONFIG - VM_SHUTDOWN_ORDER = 
2020-02-22 03:50:11 -- info: CONFIG - VM_STARTUP_ORDER = 
2020-02-22 03:50:11 -- info: CONFIG - RSYNC_LINK = 0
2020-02-22 03:50:11 -- info: CONFIG - BACKUP_FILES_CHMOD = 
2020-02-22 03:50:11 -- info: CONFIG - EMAIL_LOG = 1
2020-02-22 03:50:11 -- info: CONFIG - EMAIL_SERVER = 192.168.21.20
2020-02-22 03:50:11 -- info: CONFIG - EMAIL_SERVER_PORT = 25
2020-02-22 03:50:11 -- info: CONFIG - EMAIL_DELAY_INTERVAL = 1
2020-02-22 03:50:11 -- info: CONFIG - EMAIL_FROM = ghettoVCB@client.tld
2020-02-22 03:50:11 -- info: CONFIG - EMAIL_TO = whatever@whereever.tld
2020-02-22 03:50:11 -- info: CONFIG - WORKDIR_DEBUG = 0
2020-02-22 03:50:11 -- info: CONFIG - ENABLE NFS IO HACK = 0

2020-02-22 03:50:13 -- dryrun: ###############################################
2020-02-22 03:50:13 -- dryrun: Virtual Machine: acctsrv2
2020-02-22 03:50:13 -- dryrun: VM_ID: 6
2020-02-22 03:50:13 -- dryrun: VMX_PATH: /vmfs/volumes/localstore0/acctsrv2/acctsrv2.vmx
2020-02-22 03:50:13 -- dryrun: VMX_DIR: /vmfs/volumes/localstore0/acctsrv2
2020-02-22 03:50:13 -- dryrun: VMX_CONF: acctsrv2/acctsrv2.vmx
2020-02-22 03:50:13 -- dryrun: VMFS_VOLUME: localstore0
2020-02-22 03:50:13 -- dryrun: VMDK(s): 
2020-02-22 03:50:13 -- dryrun:  sysdrive.vmdk   80 GB
2020-02-22 03:50:13 -- dryrun: INDEPENDENT VMDK(s): 
2020-02-22 03:50:13 -- dryrun: TOTAL_VM_SIZE_TO_BACKUP: 80 GB
2020-02-22 03:50:13 -- dryrun: ###############################################

2020-02-22 03:50:13 -- info: ###### Final status: ERROR: No VMs backed up! ######

2020-02-22 03:50:13 -- info: ============================== ghettoVCB LOG END ================================

not using option -e results in almost-correct operation. I say almost-correct because VMs that always fail to backup, and create false-positives in the monitoring process, are selected for backup.

[root@esxi0:~] /vmfs/volumes/backupstore0/ghettoVCB/ghettoVCB.sh -a -g /vmfs/volumes/backupstore0/ghettoVCB/ghettoVCB.conf  -d dryrun
Logging output to "/tmp/ghettoVCB-2020-02-22_04-27-51-2212611.log" ...
2020-02-22 04:27:52 -- info: ============================== ghettoVCB LOG START ==============================

2020-02-22 04:27:52 -- info: CONFIG - USING GLOBAL GHETTOVCB CONFIGURATION FILE = /vmfs/volumes/backupstore0/ghettoVCB/ghettoVCB.conf
2020-02-22 04:27:52 -- info: CONFIG - VERSION = 2019_01_06_4
2020-02-22 04:27:52 -- info: CONFIG - GHETTOVCB_PID = 2212611
2020-02-22 04:27:52 -- info: CONFIG - VM_BACKUP_VOLUME = /vmfs/volumes/backupstore0/_ghettobackups
2020-02-22 04:27:52 -- info: CONFIG - ENABLE_NON_PERSISTENT_NFS = 0
2020-02-22 04:27:52 -- info: CONFIG - VM_BACKUP_ROTATION_COUNT = 3
2020-02-22 04:27:52 -- info: CONFIG - VM_BACKUP_DIR_NAMING_CONVENTION = 2020-02-22_04-27-51
2020-02-22 04:27:52 -- info: CONFIG - DISK_BACKUP_FORMAT = thin
2020-02-22 04:27:52 -- info: CONFIG - POWER_VM_DOWN_BEFORE_BACKUP = 0
2020-02-22 04:27:52 -- info: CONFIG - ENABLE_HARD_POWER_OFF = 0
2020-02-22 04:27:52 -- info: CONFIG - ITER_TO_WAIT_SHUTDOWN = 3
2020-02-22 04:27:52 -- info: CONFIG - POWER_DOWN_TIMEOUT = 5
2020-02-22 04:27:52 -- info: CONFIG - SNAPSHOT_TIMEOUT = 15
2020-02-22 04:27:52 -- info: CONFIG - LOG_LEVEL = dryrun
2020-02-22 04:27:52 -- info: CONFIG - BACKUP_LOG_OUTPUT = /tmp/ghettoVCB-2020-02-22_04-27-51-2212611.log
2020-02-22 04:27:52 -- info: CONFIG - ENABLE_COMPRESSION = 0
2020-02-22 04:27:52 -- info: CONFIG - VM_SNAPSHOT_MEMORY = 0
2020-02-22 04:27:52 -- info: CONFIG - VM_SNAPSHOT_QUIESCE = 0
2020-02-22 04:27:52 -- info: CONFIG - ALLOW_VMS_WITH_SNAPSHOTS_TO_BE_BACKEDUP = 0
2020-02-22 04:27:52 -- info: CONFIG - VMDK_FILES_TO_BACKUP = all
2020-02-22 04:27:52 -- info: CONFIG - VM_SHUTDOWN_ORDER = 
2020-02-22 04:27:52 -- info: CONFIG - VM_STARTUP_ORDER = 
2020-02-22 04:27:52 -- info: CONFIG - RSYNC_LINK = 0
2020-02-22 04:27:52 -- info: CONFIG - BACKUP_FILES_CHMOD = 
2020-02-22 04:27:52 -- info: CONFIG - EMAIL_LOG = 1
2020-02-22 04:27:52 -- info: CONFIG - EMAIL_SERVER = 192.168.21.20
2020-02-22 04:27:52 -- info: CONFIG - EMAIL_SERVER_PORT = 25
2020-02-22 04:27:52 -- info: CONFIG - EMAIL_DELAY_INTERVAL = 1
2020-02-22 04:27:52 -- info: CONFIG - EMAIL_FROM = ghettoVCB@client.tld
2020-02-22 04:27:52 -- info: CONFIG - EMAIL_TO = whatever@whereever.tld
2020-02-22 04:27:52 -- info: CONFIG - WORKDIR_DEBUG = 0
2020-02-22 04:27:52 -- info: CONFIG - ENABLE NFS IO HACK = 0

2020-02-22 04:27:54 -- dryrun: ###############################################
2020-02-22 04:27:54 -- dryrun: Virtual Machine: acctsrv2
2020-02-22 04:27:54 -- dryrun: VM_ID: 6
2020-02-22 04:27:54 -- dryrun: VMX_PATH: /vmfs/volumes/localstore0/acctsrv2/acctsrv2.vmx
2020-02-22 04:27:54 -- dryrun: VMX_DIR: /vmfs/volumes/localstore0/acctsrv2
2020-02-22 04:27:54 -- dryrun: VMX_CONF: acctsrv2/acctsrv2.vmx
2020-02-22 04:27:54 -- dryrun: VMFS_VOLUME: localstore0
2020-02-22 04:27:54 -- dryrun: VMDK(s): 
2020-02-22 04:27:54 -- dryrun:  sysdrive.vmdk   80 GB
2020-02-22 04:27:54 -- dryrun: INDEPENDENT VMDK(s): 
2020-02-22 04:27:54 -- dryrun: TOTAL_VM_SIZE_TO_BACKUP: 80 GB
2020-02-22 04:27:54 -- dryrun: ###############################################

2020-02-22 04:27:55 -- info: ###### Final status: OK, only a dryrun. ######

2020-02-22 04:27:55 -- info: ============================== ghettoVCB LOG END ================================

Finally, triggering the usage output does not mention the -e option, except in the examples.

[root@esxi0:~] /vmfs/volumes/backupstore0/ghettoVCB/ghettoVCB.sh 
###############################################################################
#
# ghettoVCB for ESX/ESXi 3.5, 4.x+, 5.x & 6.x
# Author: William Lam
# http://www.virtuallyghetto.com/
# Documentation: http://communities.vmware.com/docs/DOC-8760
# Created: 11/17/2008
# Last modified: 2019_01_06 Version 4
#
###############################################################################

Usage: ghettoVCB.sh [options]

OPTIONS:
   -a     Backup all VMs on host
   -f     List of VMs to backup
   -m     Name of VM to backup (overrides -f)
   -c     VM configuration directory for VM backups
   -g     Path to global ghettoVCB configuration file
   -l     File to output logging
   -w     ghettoVCB work directory (default: /tmp/ghettoVCB.work)
   -d     Debug level [info|debug|dryrun] (default: info)

(e.g.)

Backup VMs stored in a list
    /vmfs/volumes/backupstore0/ghettoVCB/ghettoVCB.sh -f vms_to_backup

Backup a single VM
    /vmfs/volumes/backupstore0/ghettoVCB/ghettoVCB.sh -m vm_to_backup

Backup all VMs residing on this host
    /vmfs/volumes/backupstore0/ghettoVCB/ghettoVCB.sh -a

Backup all VMs residing on this host except for the VMs in the exclusion list
    /vmfs/volumes/backupstore0/ghettoVCB/ghettoVCB.sh -a -e vm_exclusion_list

Backup VMs based on specific configuration located in directory
    /vmfs/volumes/backupstore0/ghettoVCB/ghettoVCB.sh -f vms_to_backup -c vm_backup_configs

Backup VMs using global ghettoVCB configuration file
    /vmfs/volumes/backupstore0/ghettoVCB/ghettoVCB.sh -f vms_to_backup -g /global/ghettoVCB.conf

Output will log to /tmp/ghettoVCB.log (consider logging to local or remote datastore to persist logs)
    /vmfs/volumes/backupstore0/ghettoVCB/ghettoVCB.sh -f vms_to_backup -l /vmfs/volume/local-storage/ghettoVCB.log

Dry run (no backup will take place)
    /vmfs/volumes/backupstore0/ghettoVCB/ghettoVCB.sh -f vms_to_backup -d dryrun

2020-02-22 04:32:58 -- info: ERROR: Incorrect number of arguments!

Is the -e option somehow deprecated? Or is this something to do with 6.7U3? I should note that we have been relying on it for years

I have not confirmed yet whether the error occurs in a normal production run, and I won't be able to find that out for a day or so (when the regularly scheduled backup run happens), due to competing demands on the target (backup) server as we re-configure things.

Thanks in advance for any help.

mark-meier commented 3 years ago

I have experienced a similar problem whereby I use the -f option to select certain VMware images to backup, while also using the -d dryrun option and an almost identical output stream (after "ghettoVCB LOG START " shows "info: CONFIG" in the config section and "dryrun" in the virtual machine section output. The final status shows "###### Final status: ERROR: No VMs backed up! ######"

If all configuration parameters are correct and the backup (with dryrun) works properly, I believe it should show "###### Final status: OK, only a dryrun. ######" (similar to other dryruns that work correctly).