msgpack.exceptions.UnpackValueError

[PiQuer]

Attic worked fine for some time, now suddenly I get this exception when trying to create or delete an archive. Arch linux, attic-0.16, python-msgpack 0.4.7 (and tried 0.4.6 without success). Backup to amazon s3 mounted with s3fs.

attic check --repair runs through without error.

Any way to recover other than recreating the repository?

Traceback (most recent call last):
  File "/usr/bin/attic", line 3, in <module>
    main()
  File "/usr/lib/python3.5/site-packages/attic/archiver.py", line 730, in main
    exit_code = archiver.run(sys.argv[1:])
  File "/usr/lib/python3.5/site-packages/attic/archiver.py", line 720, in run
    return args.func(args)
  File "/usr/lib/python3.5/site-packages/attic/archiver.py", line 129, in do_create
    self._process(archive, cache, args.excludes, args.exclude_caches, skip_inodes, path, restrict_dev)
  File "/usr/lib/python3.5/site-packages/attic/archiver.py", line 178, in _process
    os.path.join(path, filename), restrict_dev)
  File "/usr/lib/python3.5/site-packages/attic/archiver.py", line 164, in _process
    archive.process_file(path, st, cache)
  File "/usr/lib/python3.5/site-packages/attic/archive.py", line 416, in process_file
    chunks.append(cache.add_chunk(self.key.id_hash(chunk), chunk, self.stats))
  File "/usr/lib/python3.5/site-packages/attic/cache.py", line 229, in add_chunk
    self.repository.put(id, data, wait=False)
  File "/usr/lib/python3.5/site-packages/attic/repository.py", line 348, in put
    self.prepare_txn(self.get_transaction_id())
  File "/usr/lib/python3.5/site-packages/attic/repository.py", line 150, in prepare_txn
    hints = read_msgpack(os.path.join(self.path, 'hints.%d' % transaction_id))
  File "/usr/lib/python3.5/site-packages/attic/helpers.py", line 499, in read_msgpack
    return msgpack.unpack(fd)
  File "msgpack/_unpacker.pyx", line 164, in msgpack._unpacker.unpack (msgpack/_unpacker.cpp:164)
  File "msgpack/_unpacker.pyx", line 146, in msgpack._unpacker.unpackb (msgpack/_unpacker.cpp:146)
msgpack.exceptions.UnpackValueError: Unpack failed: error = 0

[ThomasWaldmann]

It looks like there is some problem with the "hints.xxx" file (see in your repo directory). Maybe it is somehow corrupted (empty or bad contents), you could try removing it.

No guarantees. If it is important, make a backup of the repo before experimenting.

[PiQuer]

Thanks for the reply. Deleting the hints.xxx file and running attic check --repair afterwards indeed fixed the problem.

I will keep an eye on the repository if the file gets corrupted again, and I'm glad to provide more information in case someone wants to debug this scenario.

Maybe in such a case (corrupted hints file) a more helpful message could be generated?

[PiQuer]

The issue keeps coming back from time to time, I do not know by what it is triggered. In these cases the hints file is empty and I have to delete it and run check --repair.

[ThomasWaldmann]

Maybe it is s3fs not running reliably / well enough as a filesystem?

Can you reproduce on a local ext4 mount?

[PiQuer]

It might very well be related to s3fs, never had it with a smb or nfs mount (other backup locations I use).