Invalid CVMFS options are generated when using <default_repositories> or wildcard matching

[tiradani]

The CVMFS options are set incorrectly when using wildcard matching for CVMFS repositories. Specifically, the timeout parameter has the repo name munged with the parameter name. So, instead of timeout=300, I see cms.cern.chtimeout=300. This produces a CVMFS error.

To reporduce (tcsh): setenv PARROT_CVMFS_REPO "*:quota_limit=4000,quota_threshold=2000,proxies=http://proxy1.fqdn:3128|http://proxy2.fqdn:3128" setenv CVMFS_TEST_PATH "/cvmfs/cms.cern.ch/cmsset_default.sh" ./parrot_run -d cvmfs -t "/dev/shm/parrot" test -e $CVMFS_TEST_PATH

The full output (with the fqdns of institutional machines removed):

2014/12/22 08:47:06.80 parrot_run[18126] child:18127 cvmfs: Using libcvmfs version: 1 2014/12/22 08:47:06.80 parrot_run[18126] child:18127 cvmfs: setenv CERNVM_UUID=`parrot-4-3-2' 2014/12/22 08:47:06.80 parrot_run[18126] child:18127 notice: Disabling alien cache since it is mutually exclusive with quota limits. 2014/12/22 08:47:06.80 parrot_run[18126] child:18127 cvmfs: filesystem configured * with repo path and options repo_name=,cachedir=/dev/shm/parrot/cvmfs/,timeout=300,timeout_direct=300,quota_limit=4000,quota_threshold=2000,proxies=http://proxy1.fqdn:3128|http://proxy2.fqdn:3128 2014/12/22 08:47:06.80 parrot_run[18126] child:18127 cvmfs: filesystem configured from pattern: cms.cern.ch with repo path and options repo_name=cms.cern.ch,cachedir=/dev/shm/parrot/cvmfs/cms.cern.ch,cms.cern.chtimeout=300,timeout_direct=300,quota_limit=4000,quota_threshold=2000,proxies=http://proxy1.fqdn:3128|http://proxy2.fqdn:3128 2014/12/22 08:47:06.80 parrot_run[18126] child:18127 cvmfs: activating repository cms.cern.ch 2014/12/22 08:47:06.80 parrot_run[18126] child:18127 cvmfs: checking lock file /dev/shm/parrot/cvmfs/cms.cern.ch/lock.cms.cern.ch 2014/12/22 08:47:06.80 parrot_run[18126] child:18127 cvmfs: cvmfs_init(repo_name=cms.cern.ch,cachedir=/dev/shm/parrot/cvmfs/cms.cern.ch,cms.cern.chtimeout=300,timeout_direct=300,quota_limit=4000,quota_threshold=2000,proxies=http://proxy1.fqdn:3128|http://proxy2.fqdn:3128) Unknown libcvmfs option: cms.cern.chtimeout Invalid CVMFS options: repo_name=cms.cern.ch,cachedir=/dev/shm/parrot/cvmfs/cms.cern.ch,cms.cern.chtimeout=300,timeout_direct=300,quota_limit=4000,quota_threshold=2000,proxies=http://proxy1.fqdn:3128|http://proxy2.fqdn:3128. CernVM-FS version 2.1.19 Copyright (c) 2009- CERN All rights reserved

Please visit http://cernvm.cern.ch/project/info for license details and author list.

libcvmfs options are expected in the form: option1,option2,option3,... Within an option, the characters , and \ must be preceded by .

options are: url=REPOSITORY_URL The URL of the CernVM-FS server(s): 'url1;url2;...' timeout=SECONDS Timeout for network operations (default is 2) timeout_direct=SECONDS Timeout for network operations without proxy (default is 2) cachedir=DIR Where to store disk cache alien_cachedir=DIR Store data chunks separately proxies=HTTP_PROXIES Set the HTTP proxy list, such as 'proxy1|proxy2;DIRECT' tracefile=FILE Trace FUSE opaerations into FILE pubkey=PEMFILE Public RSA key that is used to verify the whitelist signature. allow_unsigned Accept unsigned catalogs (allows man-in-the-middle attacks) rebuild_cachedb Force rebuilding the quota cache db from cache directory quota_limit=MB Limit size of data chunks in cache. -1 Means unlimited. quota_threshold=MB Cleanup until size is <= threshold nofiles=NUMBER Set the maximum number of open files for CernVM-FS process (soft limit) logfile=FILE Logs all messages to FILE instead of stderr and daemonizes. Makes only sense for the debug version deep_mount=prefix Path prefix if a repository is mounted on a nested catalog, i.e. deep_mount=/software/15.0.1 repo_name= Unique name of the mounted repository, e.g. atlas.cern.ch mountpoint= Path to root of repository, e.g. /cvmfs/atlas.cern.ch blacklist=FILE Local blacklist for invalid certificates. Has precedence over the whitelist. syslog_level=NUMBER Sets the level used for syslog to DEBUG (1), INFO (2), or NOTICE (3). Default is NOTICE. Note: you cannot load files greater than quota_limit-quota_threshold 2014/12/22 08:47:06.80 parrot_run[18126] child:18127 cvmfs: activating repository cms.cern.ch 2014/12/22 08:47:06.80 parrot_run[18126] child:18127 cvmfs: checking lock file /dev/shm/parrot/cvmfs/cms.cern.ch/lock.cms.cern.ch 2014/12/22 08:47:06.80 parrot_run[18126] child:18127 cvmfs: cvmfs_init(repo_name=cms.cern.ch,cachedir=/dev/shm/parrot/cvmfs/cms.cern.ch,cms.cern.chtimeout=300,timeout_direct=300,quota_limit=4000,quota_threshold=2000,proxies=http://proxy1.fqdn:3128|http://proxy2.fqdn:3128) Unknown libcvmfs option: cms.cern.chtimeout Invalid CVMFS options: repo_name=cms.cern.ch,cachedir=/dev/shm/parrot/cvmfs/cms.cern.ch,cms.cern.chtimeout=300,timeout_direct=300,quota_limit=4000,quota_threshold=2000,proxies=http://proxy1.fqdn:3128|http://proxy2.fqdn:3128. CernVM-FS version 2.1.19 Copyright (c) 2009- CERN All rights reserved

Please visit http://cernvm.cern.ch/project/info for license details and author list.

libcvmfs options are expected in the form: option1,option2,option3,... Within an option, the characters , and \ must be preceded by .

options are: url=REPOSITORY_URL The URL of the CernVM-FS server(s): 'url1;url2;...' timeout=SECONDS Timeout for network operations (default is 2) timeout_direct=SECONDS Timeout for network operations without proxy (default is 2) cachedir=DIR Where to store disk cache alien_cachedir=DIR Store data chunks separately proxies=HTTP_PROXIES Set the HTTP proxy list, such as 'proxy1|proxy2;DIRECT' tracefile=FILE Trace FUSE opaerations into FILE pubkey=PEMFILE Public RSA key that is used to verify the whitelist signature. allow_unsigned Accept unsigned catalogs (allows man-in-the-middle attacks) rebuild_cachedb Force rebuilding the quota cache db from cache directory quota_limit=MB Limit size of data chunks in cache. -1 Means unlimited. quota_threshold=MB Cleanup until size is <= threshold nofiles=NUMBER Set the maximum number of open files for CernVM-FS process (soft limit) logfile=FILE Logs all messages to FILE instead of stderr and daemonizes. Makes only sense for the debug version deep_mount=prefix Path prefix if a repository is mounted on a nested catalog, i.e. deep_mount=/software/15.0.1 repo_name= Unique name of the mounted repository, e.g. atlas.cern.ch mountpoint= Path to root of repository, e.g. /cvmfs/atlas.cern.ch blacklist=FILE Local blacklist for invalid certificates. Has precedence over the whitelist. syslog_level=NUMBER Sets the level used for syslog to DEBUG (1), INFO (2), or NOTICE (3). Default is NOTICE. Note: you cannot load files greater than quota_limit-quota_threshold

I am guessing that there is a problem in the createMatch function in pfs_service_cvmfs.cc but I haven't been able to pin it down yet.

[dthain]

Yep, that's a bug, looking into it...

[dthain]

This was definitely a bug in how we construct the configuration string, but apparently only cropped up when quotas were enabled. You can pull the fixed version from our repo, or download a build here: http://ccl.cse.nd.edu/software/autobuild

Please let us know if that fixes the problem.