"git-annex: Cannot run git-annex-remote-b2"

[gjost]

pkikawa 2018-03-23 16:31:

Environment:
Request Method: GET
Request URL: http://192.168.56.101/ui/file/ddr-densho-332-2-mezzanine-3f1a9ba496/
Django Version: 1.7.11
Python Version: 2.7.13
Installed Applications:
('django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'bootstrap_pagination',
 'djcelery',
 'gunicorn',
 'rest_framework',
 'sorl.thumbnail',
 'ddrlocal',
 'storage',
 'webui')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware')
Traceback:
File "/opt/ddr-local/venv/ddrlocal/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  111.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/opt/ddr-local/ddrlocal/storage/decorators.py" in inner
  84.         return func(request, *args, **kwargs)
File "/opt/ddr-local/ddrlocal/webui/views/files.py" in detail
  114.         repository(collection.path_abs), file_
File "/opt/ddr-local/ddrlocal/webui/gitstatus.py" in annex_whereis_file
  104.             annex_info(repo)
File "/opt/ddr-local/venv/ddrlocal/lib/python2.7/site-packages/ddr_cmdln-0.9.4b0-py2.7.egg/DDR/dvcs.py" in annex_whereis_file
  988.     data = json.loads(repo.git.annex('whereis', '--json', file_path_rel))
File "/opt/ddr-local/venv/ddrlocal/lib/python2.7/site-packages/git/cmd.py" in <lambda>
  440.         return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
File "/opt/ddr-local/venv/ddrlocal/lib/python2.7/site-packages/git/cmd.py" in _call_process
  834.             return self.execute(make_call(), **_kwargs)
File "/opt/ddr-local/venv/ddrlocal/lib/python2.7/site-packages/git/cmd.py" in execute
  627.                 raise GitCommandError(command, status, stderr_value)
Exception Type: GitCommandError at /ui/file/ddr-densho-332-2-mezzanine-3f1a9ba496/
Exception Value: 'git annex whereis --json files/ddr-densho-332-2/files/ddr-densho-332-2-mezzanine-3f1a9ba496.tif' returned with exit code 1
stderr: 'git-annex: Cannot run git-annex-remote-b2 -- Make sure it's in your PATH and is executable.'

[gjost]

This is happening on the detail page for a mezzanine binary, in the code that produces the list of remote locations of the file:

Trusted remotes
hq-shenandoah   89b6858d-247f-4343-b49c-784900dd8c3d
ddr@ddr-editor:/media/qnfs/kinkura/gold/ddr-densho-332  a4d84966-2048-4b01-9327-eb682d00de7c
cloud-backblaze-b2  bd80b391-5f37-4e57-814c-5a614587b2e6

[gjost]

On the affected machine:

densho@ddreditor:~$ sudo git annex version
[sudo] password for densho:
git-annex version: 6.20170101.1
build flags: Assistant Webapp Pairing Testsuite S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify ConcurrentOutput TorrentParser MagicMime Feeds Quvi
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav tahoe glacier ddar hook external

[gjost]

GFroh: looks like editor is looking for the b2 annex extension app b/c the repo on kinkura has a b2 spec remote.

[gjost]

FWIW I cloned that repo on my local. I can see the backblaze remote in the repo's list of remotes but i dont get the error.

[gjost]

GFroh: that’s b/c the special remote is not connected to your repo. looks like joeyh has a way of telling git annex to ignore certain remotes by default; but that functionality is in a slightly newer version than what we’re running (http://git-annex.branchable.com/forum/Is_there_an___39__annex-cost__39___to_NEVER_access_remote__63__/) (‘annex-ignore’) we’re going to see what happens if the b2 extension is available, but without credentials defined so it won’t actually be able to make the network call. if git-annex just reports an error trying to reach that remote, but doesn’t throw a full exception (and properly checks whereis status for the other remotes) then maybe we can just call it good.

[gjost]

Add config file setting to optionally disable the git-annex-whereis call.

[GeoffFroh]

We'll work at HQ to redesign the backup infrastructure so that the b2 special remotes are attached to repo clones outside of gold

[gjost]

I added a new config setting that controls whether file-detail runs git-annex-wheres:

[local]
git_annex_whereis=True

Set it to False and restart to disable whereis.

[pkikawa]

so on 2.2.7 i got binaries to work but no dice on master. i have git_annex_whereis=False in both the ddrlocal.cfg and ddrlocal-local.cfg

[pkikawa]

tried setting it to false even in the ddrlocal.cfg.dpkg-dist to no avail.

[gjost]

Did you restart the app? sudo service supervisor restart

[gjost]

ddr-cmdln and ddr-local only look at config files ending in .cfg - they would ignore ddrlocal.cfg.dpkg-dist.

I've never even seen a ddrlocal.cfg.dpkg-dist file. My guess is that it's something the Debian packaging system (dpkg is a component of this) put in while installing, probably if it was trying to install ddrlocal.cfg but it was already there or something. (For instance, if I'm replacing or editing a system config file I often make a copy of the original called SOMEFILE.conf.orig)

You might have gotten that if you removed ddrlocal-develop and installed ddrlocal-master on a system. apt-get remove uninstalls the specified app but usually leaves config files in place. IIRC apt-get purge the configs and any other files associated with a package.

Update/Correction: The packager should ask what you want to do with an existing config file. The default behavior is to leave whatever is already there in place, and to install the package's version as FILE.dpkg-dist in the same directory. So you probably want to do this:

cd /etc/ddr; sudo mv ddrlocal.cfg.dpkg-dist ddrlocal.cfg`

and then restart.