search

denshoproject / ddr-cmdln

Command-line tools for automating the Densho Digital Repository's various processes.
Other
0 stars 2 forks source link

git-annex: Repository version 8 is not supported. #194

Closed gjost closed 3 years ago

gjost commented 3 years ago

@pkikawa reports this error from a number of collections (densho-420, 419, 415) and on multiple workstations (akoyet and entiat)

Traceback (most recent call last):
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/ddr-local/ddrlocal/storage/decorators.py", line 83, in inner
    return func(request, *args, **kwargs)
  File "/opt/ddr-local/ddrlocal/webui/views/collections.py", line 93, in detail
    'annex_info': annex_info(repository(collection.path_abs)),
  File "/opt/ddr-local/ddrlocal/webui/gitstatus.py", line 89, in annex_info
    data = dvcs.annex_info(repo)
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/ddr_cmdln-3.0.0.post1-py3.7.egg/DDR/dvcs.py", line 1059, in annex_info
    data = json.loads(repo.git.annex('info', '--fast', '--json'))
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/git/cmd.py", line 542, in <lambda>
    return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/git/cmd.py", line 1005, in _call_process
    return self.execute(call, **exec_kwargs)
  File "/opt/ddr-local/venv/ddrlocal/lib/python3.7/site-packages/git/cmd.py", line 822, in execute
    raise GitCommandError(command, status, stderr_value, stdout_value)
Exception Type: GitCommandError at /ui/collection/ddr-densho-420/
Exception Value: Cmd('git') failed due to: exit code(1)
  cmdline: git annex info --fast --json
  stderr: 'git-annex: Repository version 8 is not supported. Upgrade git-annex.'

Error occurs on collection detail page. ddr-cmdln-194-GitCommandError.zip

gjost commented 3 years ago

@pkikawa What Linux versions are used on akoyet and entiat?

gjost commented 3 years ago

Best way to find out: cat /etc/debian_version

pkikawa commented 3 years ago

both are deb 10 (akoyet vm is a clone from entiat)

pkikawa commented 3 years ago

also found densho-377 throwing the same error

gjost commented 3 years ago

Can you get me the version of git-annex on these machines and the versions of some of the affected of the repositories?

git annex version
cd /PATH/TO/REPOSITORY; git annex info
pkikawa commented 3 years ago

(ddrlocal) ddr@ddrlocal:/media/qnfs/kinkura/gold/ddr-densho-420$ git annex version git-annex version: 7.20190129 build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite dependency versions: aws-0.20 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.3 feed-1.0.0.0 ghc-8.4.4 http-client-0.5.13.1 persistent-sqlite-2.8.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0 key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external operating system: linux x86_64 supported repository versions: 5 7 upgrade supported from repository versions: 0 1 2 3 4 5 6 local repository version: 8 (ddrlocal) ddr@ddrlocal:/media/qnfs/kinkura/gold/ddr-densho-420$ ^C (ddrlocal) ddr@ddrlocal:/media/qnfs/kinkura/gold/ddr-densho-420$ git annex info git-annex: Repository version 8 is not supported. Upgrade git-annex. (ddrlocal) ddr@ddrlocal:/media/qnfs/kinkura/gold/ddr-densho-420$

pkikawa commented 3 years ago

(ddrlocal) ddr@ddrlocal:/media/qnfs/kinkura/gold/ddr-densho-377$ git annex version git-annex version: 7.20190129 build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite dependency versions: aws-0.20 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.3 feed-1.0.0.0 ghc-8.4.4 http-client-0.5.13.1 persistent-sqlite-2.8.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0 key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external operating system: linux x86_64 supported repository versions: 5 7 upgrade supported from repository versions: 0 1 2 3 4 5 6 local repository version: 8 (ddrlocal) ddr@ddrlocal:/media/qnfs/kinkura/gold/ddr-densho-377$ git annex info git-annex: Repository version 8 is not supported. Upgrade git-annex. (ddrlocal) ddr@ddrlocal:/media/qnfs/kinkura/gold/ddr-densho-377$

pkikawa commented 3 years ago

(ddrlocal) ddr@ddrlocal:/media/qnfs/kinkura/gold/ddr-densho-419$ git annex version git-annex version: 7.20190129 build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite dependency versions: aws-0.20 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.3 feed-1.0.0.0 ghc-8.4.4 http-client-0.5.13.1 persistent-sqlite-2.8.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0 key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external operating system: linux x86_64 supported repository versions: 5 7 upgrade supported from repository versions: 0 1 2 3 4 5 6 local repository version: 8 (ddrlocal) ddr@ddrlocal:/media/qnfs/kinkura/gold/ddr-densho-419$ git annex info git-annex: Repository version 8 is not supported. Upgrade git-annex. (ddrlocal) ddr@ddrlocal:/media/qnfs/kinkura/gold/ddr-densho-419$

gjost commented 3 years ago

Looks like someone used an Ubuntu 20.04 or a Debian Testing machine to interact with these collection repos. This resulted in the repos being upgraded to git-annex version 8-something.

pkikawa commented 3 years ago

(ddrlocal) ddr@ddrlocal:/media/qnfs/kinkura/gold/ddr-densho-415$ git annex version git-annex version: 7.20190129 build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite dependency versions: aws-0.20 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.3 feed-1.0.0.0 ghc-8.4.4 http-client-0.5.13.1 persistent-sqlite-2.8.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0 key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external operating system: linux x86_64 supported repository versions: 5 7 upgrade supported from repository versions: 0 1 2 3 4 5 6 local repository version: 8 (ddrlocal) ddr@ddrlocal:/media/qnfs/kinkura/gold/ddr-densho-415$ git annex info git-annex: Repository version 8 is not supported. Upgrade git-annex. (ddrlocal) ddr@ddrlocal:/media/qnfs/kinkura/gold/ddr-densho-415$

gjost commented 3 years ago

@pkikawa Can you get version info for shenandoah pls?

cat /etc/os-release
git annex version
pkikawa commented 3 years ago

NAME="Ubuntu" VERSION="20.04.1 LTS (Focal Fossa)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 20.04.1 LTS" VERSION_ID="20.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=focal UBUNTU_CODENAME=focal

gjost commented 3 years ago

I'm putting together changes to Makefile to install git and git-annex from Debian Backports (https://packages.debian.org/buster-backports/git-annex) which will a similar version to Ubuntu 20.04 LTS. Unfortunately right now I can't see a way to to make our packaging tool FPM install things from backports.

pkikawa commented 3 years ago

also densho-287

gjost commented 3 years ago

OK I have a temporary fix. ddr-cmdln commit 5a3d8e2 on the develop branch changes Makefile to install git and git-annex from backports.

Please install this only on machines that need to work with the affected collection repos, and please ask the users to only use this VM to work on those collections so we don't affect any more. I'd really like to roll out a version change like this in a controlled fashion rather than how we're doing it right now.

Here's how to upgrade:

cd /opt/ddr-cmdln
sudo git checkout develop
sudo git pull
sudo make install
gjost commented 3 years ago

Our packaging tool, FPM (https://fpm.readthedocs.io/) doesn't have a built-in way to install backports so I'm working on a pre-install script to add the buster-backports repository and install git-annex and git.

gjost commented 3 years ago

Fixed in ddr-cmdln commit 3d72aa6 for package ddrcmdln-master 5.0.16~deb10, and ddr-local commit 3ef87f1 for package ddrlocal-master 5.0.16~deb10.

This is not an ideal solution. I couldn't find a way to both set up the Debian backports repository and install the packages within the package install process. Instead the install procedure will go like this:

sudo gdebi ddrlocal-master 5.0.16~deb10
cd /opt/ddr-local
sudo make install-git
pkikawa commented 3 years ago

tested and fixed