backup does throw a fatal error when we have a symbolic link under the alf_data folder

Hello Tony,

We have stored our alfresco contentstore on a NAS server and linked to that data with symbolic links in the alf_data folder.

root@Alfresco-test:/opt/alfresco/scripts# cd /opt/alfresco/alf_data/ root@Alfresco-test:/opt/alfresco/alf_data# ll drwxr-xr-x 2 alfresco-db-backup/ lrwxrwxrwx 1 contentstore -> /mnt/Transport/alfresco_data_storage/contentstore/ lrwxrwxrwx 1 contentstore.deleted -> /mnt/Transport/alfresco_data_storage/contentstore.deleted// lrwxrwxrwx 1 keystore -> /mnt/Transport/alfresco_data_storage/keystore// drwx------ 12 postgresql/ lrwxrwxrwx 1 solr -> /mnt/Transport/alfresco_data_storage/solr// drwxr-xr-x 4 solrBackup/

The ALF_CONTENTSTORE property is set to the default value ${ALF_DIRROOT}/contentstore and then the backup is run (backup all).

Unfortunately nothing gets written in the backup. Then I changed the ALF_CONTENTSTORE property to point to the real location of the data (/mnt/Transport/alfresco_data_storage/contentstore) and start again.

Then alfresco-bart throws the following exception:

root@Alfresco-test:/opt/alfresco/scripts# ./alfresco-bart.sh backup cs Fatal Error: Backup source directory has changed. Current directory: /mnt/Transport/alfresco_data_storage Previous directory: /opt/alfresco-4.1.1.3/alf_data

Aborting because you may have accidentally tried to backup two different data sets to the same remote location, or using the same archive directory. If this is not a mistake, use the --allow-source-mismatch switch to avoid seeing this message

Ok I try the allow-source-mismatch option, but that does not exists so I used the force parameter.

root@Alfresco-test:/opt/alfresco/scripts# ./alfresco-bart.sh backup all force

Then the following fatal error is shown (after the backup process i think) Fatal Error: The file specification /mnt/Transport/alfresco_data_storage/contentstore cannot match any files in the base directory /opt/alfresco-4.1.1.3 Useful file specifications begin with the base directory or some pattern (such as '**') which matches the base directory. root@Alfresco-test:/opt/alfresco/scripts#

Is there a way to avoid this problem? Or should we configure the alf_data structure differently?

Kind regards, Bart Berden APS Group BV

Hi Bart :D Duplicity doesn't follow symlinks by design then Alfresco BART can't do it. I would like to review your configuration, could you post here your alfresco-bart.properties file without passwords?

Sorry didn't know that limitation of duplicity. What is the best way to solve this symbolic link issue so that we can use Alfresco BART. By configuring the entire alf_data folder to be on a remote network drive?

Here is the alfresco-bart.properties file. Mostly default since only disk storage with no encryption is used.

# Copyright (c) 2013 Toni de la Fuente.
#
# This program is free software: you can redistribute it and/or modify it under
# the terms of the Apache License as published by the Apache Foundation.
#
# This program is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
# FOR A PARTICULAR PURPOSE. 
#
# Most recent information about this tool is available in:
# http://blyx.com/alfresco-bart
#
# Latest code available at:
# http://blyx.com/alfresco-bart
#
#########################################################################################
# alfresco-bart: ALFRESCO BACKUP AND RECOVERY TOOL 
# Version 0.2       
#########################################################################################
# ACTION REQUIRED: 
# this file should contains passwords, please, set this file as read only 
# for root (chmod 400 alfresco-bart.properties)
# Copy this file in you ALFRESCO_INSTALLATION_PATH/scripts.
#########################################################################################
## Logging
# Log configuration
LOG_DATE_FILES=`date +%F`
LOG_DATE_LOG=`date +%F-%X`
# Directory to left all Alfresco BART tasks logs with reports.
ALFBRT_LOG_DIR=/opt/alfresco/scripts
ALFBRT_LOG_FILE=${ALFBRT_LOG_DIR}/alfresco-bart-${LOG_DATE_FILES}.log
## Duplicity Setup ##
# If yes, please make sure you specify either PASSPHRASE, see INSTALL file.
ENCRYPTION_ENABLED=false
PASSPHRASE=alfresco_rocks
DUPLICITYBIN=`which duplicity`
GZIP=`which gzip`
## Backup policies
# Number of days of every full backup (if not backup found it does a full)
FULLDAYS=30D
# Backup policies to apply all backups collections (retention and cleanup)
BACKUP_POLICIES_ENABLED=true
# Number of moths to remove all backups older than or backup retention period
CLEAN_TIME=12M
# After MAXFULL counter, all incrementals will be deleted and all full will be kept until CLEAN_TIME applies, 
MAXFULL=6
# If you want to keep full backups of last 12 months but only with incremental in last 6 months
# you must set CLEAN_TIME=12M and MAXFULL=6
# Volume size in MB, default is 25MB per backup volume, consider reduce or increase it if you
# are doing tape backup (if a backup takes 60MB you will get 3 volumes, 25+25+10)
VOLUME_SIZE=25
# Alfresco root installation path. It has to be a real directory not a symlink
ALF_INSTALLATION_DIR=/opt/alfresco-4.1.1.3
# Alfresco alf_data path
ALF_DIRROOT=${ALF_INSTALLATION_DIR}/alf_data
# Duplicity log vervosity 0 Error, 2 Warning, 4 Notice, 8 Info, 9 Debug (noisiest)
# 0 recommended for production
DUPLICITY_LOG_VERBOSITY=4
BART_LOG_TAG=[BART-Loglevel$DUPLICITY_LOG_VERBOSITY]
GLOBAL_DUPLICITY_PARMS="-v${DUPLICITY_LOG_VERBOSITY} --volsize=${VOLUME_SIZE} --log-file=${ALFBRT_LOG_FILE} --full-if-older-than ${FULLDAYS} --asynchronous-upload"
## Indexes Configuration
BACKUP_INDEX_ENABLED=true
# Alfresco index type, use solr or lucene
INDEXTYPE=solr
# If Solr is selected, all Solr installation and config files will be included in your backup
INDEXES_DIR=${ALF_DIRROOT}/solr
# If Lucene change to ${ALF_DIRROOT}/lucene
# SOLR or Lucene backup location
INDEXES_BACKUP_DIR=${ALF_DIRROOT}/solrBackup
# If lucene add the apropiate location like ${ALF_DIRROOT}/backup-lucene-indexes
# If solr, please go to Alfresco Share Admin Console - Search Solr and set "Backups to 
# keep to 1" in both Main and Archive
# Other that do a backup of the "backed up" indexes, to easy restore of your Solr 
# indexes is recommendable to backup all configuration files but the live files,
# To enable Solr installation and configuration backup set next property
## DB Configuration ##
BACKUP_DB_ENABLED=true
# use mysql, postgresql or oracle
DBTYPE=postgresql
LOCAL_BACKUP_DB_DIR=${ALF_DIRROOT}/alfresco-db-backup
# if you are using the bundle posgresql db, with next value it will include pg live files
# for easy restore. For remote, different or external db leave it as is.
LOCAL_DB_DIR=${ALF_DIRROOT}/postgresql
DATE_FILE=`date +%Y-%m-%d_%H%M%S`
# Global DB parameters
DBNAME=alfresco
DBUSER=alfresco
DBPASS=xxxx
DBHOST=localhost
# MySQL - Alfresco DB Configuration
MYSQL_BINDIR=/usr/bin
MYSQLDUMP_BIN=mysqldump
# PostgreSQL - Alfresco DB Configuration
PGSQL_BINDIR=${ALF_INSTALLATION_DIR}/postgresql/bin
PGSQLDUMP_BIN=pg_dump
# Add PGUSER and PGPASSWORD if required for your installation and you are not doing 
# running BART as root.
PGUSER=
PGPASSWORD=${DBPASS}
# If you found problems to dump your Postgres DB try creating a file .pgpass with
# content "::::alfresco" without quotes, while alfresco is the password of your alfresco db
PGPASSFILE=${ALF_DIRROOT}/.pgpass
# Oracle - Alfresco DB Configuration
ORACLE_BINDIR=
ORASQLDUMP_BIN=exp
## ContentStore Configuration
BACKUP_CONTENTSTORE_ENABLED=true
#ALF_CONTENTSTORE=${ALF_DIRROOT}/contentstore
ALF_CONTENTSTORE=/mnt/Transport/alfresco_data_storage/contentstore
# To exclude contentstore.deleted leave next value empty
ALF_CONTENSTORE_DELETED=
ALF_CACHED_CONTENTSTORE=
ALF_CONTENTSTORE2=
ALF_CONTENTSTORE3=
ALF_CONTENTSTORE4=
ALF_CONTENTSTORE5=
## Alfresco configuration and application files backup
# This backup includes deployments and configuration files excludes 
# the ContentStore, Indexes and DB as above
BACKUP_FILES_ENABLED=true
## Backup type ##
# use s3, ftp, scp, local
BACKUPTYPE=local
## LOCAL backup destination folder ##
# absolute path starting with /
LOCAL_BACKUP_FOLDER=/mnt/backup
## Amazon S3 information ##
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
# Upper case bucket name is not allow
S3FILESYSLOCATION="s3+http://your-bucket-name"
S3OPTIONS="--s3-use-new-style --s3-use-rrs"
S3_EUROPEAN_BUCKET=false # not used yet
## FTP Information ##
FTP_SERVER=
FTP_USER=
FTP_PASSWORD=
FTP_FOLDER=
FTP_PORT=21
FTPS_ENABLE=false
## SCP Information ## 
# SSH shared keys required
SCP_SERVER=
SCP_USER=
SCP_FOLDER=
## Backup temp folder definition ##
TEMPFOLDERNAME="alfresco-backup-`date +%F-%s`"
## Recovery configuration 
ENABLE_SINGLE_FILE_RECOVERY=false
# Temporary DB for recovery and get the phisical path of a single content.
REC_DBTYPE=mysql
# Recovery database must be already created but EMPTY! Alfresco-BART will recover your
# backup database and find the files for you, after this task the recovery database 
# tables will be deleted.
# MySQL - Recovery DB Configuration ##
REC_MYDBNAME=alfresco_rec
REC_MYUSER=alfresco
REC_MYPASS=alfresco
REC_MYHOST=localhost
REC_MYSQL_BIN=/usr/bin/mysql
REC_MYSQLDUMP_BIN=/usr/bin/mysqldump
# PgSQL - Recovery DB Configuration ##
REC_PGDBNAME=alfresco_rec
REC_PGUSER=alfresco
REC_PGPASS=alfresco
REC_PGHOST=localhost
REC_PGSQL_BIN=/opt/alfresco/postgresql/bin/pgsql
REC_PGSQLDUMP_BIN=/opt/alfresco/postgresql/bin/pg_dump
# Oracle - Recovery DB Configuration ##
REC_ORADBNAME=alfresco_rec
REC_ORAUSER=alfresco
REC_ORAPASS=alfresco
REC_ORAHOST=localhost
REC_ORASQL_BIN=
REC_ORASQLDUMP_BIN=exp

@toniblyx any idea how to proceed with this problem. Change the way alf_data map is located on a remote network drive?

I am getting this error as well. Has anyone resolved it?

This is the exact error I am receiving:

Fatal Error: The file specification /opt/alf_data/solrBackup cannot match any files in the base directory /opt/alfresco-4.2.c Useful file specifications begin with the base directory or some pattern (such as '**') which matches the base directory.

I solved my problems with two changes -do not use symbolic links in the alf_data folder. Instead relocate the content_store with the dir.contentstore and dir.contentstore.deleted properties in alfresco-global.properties -the duplicity error that you are referring to was in my case that the alfresco-bart.sh script tries to backup files. (The error only occured during backup files).

The command that is executed is duplicity --include dir alf_data -- exclude location. So the alf_data map is backupped without the contentstore (that is what we want) but since the content_store is not located in the alf_data map the exclude location is not included in the include location. Hence the duplicity error.

My fix consists of simply uncommenting the two exclude lines in the alfresco-bart.sh file.

function filesBackup {
    # Getting a variable to know all includes and excludes
        FILES_DIR_INCLUDES="$ALF_INSTALLATION_DIR"
        if [ -d "$INDEXES_BACKUP_DIR" ]; then
                OPT_INDEXES_BACKUP_DIR=" --exclude $INDEXES_BACKUP_DIR"
        fi
        if [ -d "$INDEXES_DIR" ]; then
                OPT_INDEXES_DIR=" --exclude $INDEXES_DIR"
        fi
# 2014-02-25 bberden
# Disabled exclude of $ALF_CONTENTSTORE and $ALF_CONTENTSTORE_DELETED because these locations /mnt/.... are not within the INCLUDE folder /opt/alfresco/...
# Causes duplicity to fail with error:
# Fatal Error: The file specification
#    /mnt/Transport/alfresco_data_storage/contentstore
# cannot match any files in the base directory
#    /opt/alfresco-4.1.1.3
#
#       if [ -d "$ALF_CONTENTSTORE" ]; then
#               OPT_ALF_CONTENTSTORE=" --exclude $ALF_CONTENTSTORE"
#       fi
#       if [ -d ${ALF_DIRROOT}/contentstore.deleted ]; then
#               OPT_ALF_CONTENSTORE_DELETED=" --exclude ${ALF_DIRROOT}/contentstore.deleted"
#       fi
        if [ -d "$ALF_CACHED_CONTENTSTORE" ]; then
                OPT_CACHED_CONTENTSTORE=" --exclude $CACHED_CONTENTSTORE"
        fi

It is a problem when content store is not under alfresco installation folder. I have commented out the --excluding part of the filesBackup method to have it going .

@toniblyx I believe this is not a real problem with the script, right? How about closing this issue?

toniblyx / alfresco-backup-and-recovery-tool

backup does throw a fatal error when we have a symbolic link under the alf_data folder #3