Fails on Debian Wheezy 64-bit

[ChojinDSL]

Hi, I'm using Debian Wheezy 64-bit on my server. I run linux kernel 3.19 with the PF patchset. btrfs tools version 3.14.1

When I try to do anything with buttersink, it crashes with a bunch of python error messages. For example, simply calling: "buttersink /Path/To/SnapShots/"

Produces the following output.

Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/buttersink-0.6-py2.7.egg/buttersink/buttersink.py", line 214, in main with source: File "/usr/local/lib/python2.7/dist-packages/buttersink-0.6-py2.7.egg/buttersink/Store.py", line 74, in enter self._fillVolumesAndPaths(self.paths) File "/usr/local/lib/python2.7/dist-packages/buttersink-0.6-py2.7.egg/buttersink/ButterStore.py", line 67, in _fillVolumesAndPaths for bv in mount.subvolumes: File "/usr/local/lib/python2.7/dist-packages/buttersink-0.6-py2.7.egg/buttersink/btrfs.py", line 483, in subvolumes self._getRoots() File "/usr/local/lib/python2.7/dist-packages/buttersink-0.6-py2.7.egg/buttersink/btrfs.py", line 635, in _getRoots info = btrfs_root_item.read(buf) File "/usr/local/lib/python2.7/dist-packages/buttersink-0.6-py2.7.egg/buttersink/ioctl.py", line 226, in read args = list(self._struct.unpack_from(data, offset)) TypeError: unpack_from() argument 1 must be string or read-only buffer, not memoryview

Any idea what might be the cause? Off the top of my head, I would assume some libs are too old in Debian Wheezy (Stable), since I had similar problems with a btrfs dedup tool.

[AmesCornish]

I suspect it's your version of Python. Could you let me know what you get with 'python --version'?

• Ames

On Tue, Mar 17, 2015 at 10:00 AM, ChojinDSL notifications@github.com wrote:

Hi, I'm using Debian Wheezy 64-bit on my server. I run linux kernel 3.19 with the PF patchset. btrfs tools version 3.14.1

When I try to do anything with buttersink, it crashes with a bunch of python error messages. For example, simply calling: "buttersink /Path/To/SnapShots/"

Produces the following output.

Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/buttersink-0.6-py2.7.egg/buttersink/buttersink.py", line 214, in main with source: File "/usr/local/lib/python2.7/dist-packages/buttersink-0.6-py2.7.egg/buttersink/Store.py", line 74, in enter self._fillVolumesAndPaths(self.paths) File "/usr/local/lib/python2.7/dist-packages/buttersink-0.6-py2.7.egg/buttersink/ButterStore.py", line 67, in _fillVolumesAndPaths for bv in mount.subvolumes: File "/usr/local/lib/python2.7/dist-packages/buttersink-0.6-py2.7.egg/buttersink/btrfs.py", line 483, in subvolumes self._getRoots() File "/usr/local/lib/python2.7/dist-packages/buttersink-0.6-py2.7.egg/buttersink/btrfs.py", line 635, in _getRoots info = btrfs_root_item.read(buf) File "/usr/local/lib/python2.7/dist-packages/buttersink-0.6-py2.7.egg/buttersink/ioctl.py", line 226, in read args = list(self._struct.unpack_from(data, offset)) TypeError: unpack_from() argument 1 must be string or read-only buffer, not memoryview

Any idea what might be the cause? Off the top of my head, I would assume some libs are too old in Debian Wheezy (Stable), since I had similar problems with a btrfs dedup tool.

— Reply to this email directly or view it on GitHub https://github.com/AmesCornish/buttersink/issues/2.

[man0lis]

Hi, i seem to have exactly the same problem on Debian Wheezy. So in case there is no answer from ChojinDSL, here is my python version. $ python --version Python 2.7.3

So long

Edit: Kernel: 3.16.0-0.bpo.4-amd64 btrfs-progs: v3.14.1

Possible inspiration for a fix https://github.com/celery/celery/commit/574f6a545a06306e54a7be3ed8e99a5ad485d72d

[AmesCornish]

I have pushed buttersink 0.6.1 which should work around this issue on Python 2.7.3 or earlier. Please let me know if anyone is having issues.

[AmesCornish]

FYI I've released 0.6.1 to address this.

• Ames On Mar 22, 2015 10:21 AM, "Robin Nehls" notifications@github.com wrote:

My current workaround

diff --git a/buttersink/ioctl.py b/buttersink/ioctl.py index 54c808c..7c5e2bc 100644 --- a/buttersink/ioctl.py +++ b/buttersink/ioctl.py @@ -223,7 +223,10 @@ class Structure:

def read(self, data, offset=0): """ Read data structure and return (nested) named tuple(s). """

• args = list(self._struct.unpack_from(data, offset))
• if type(data).name == 'memoryview':
• args = list(self._struct.unpack_from(data.tobytes(), offset))
• else:

• args = list(self._struct.unpack_from(data, offset)) args.reverse() return self.popValue(args)

  — Reply to this email directly or view it on GitHub https://github.com/AmesCornish/buttersink/issues/2#issuecomment-84630060 .

[man0lis]

Hi, thanks for looking into this, but there seems to be another problem. running

# buttersink /mn/btrfs/snapshots

on my debain wheezy server, now works and lists subvolumes. but when I try to sync snapshots from my laptop to my server the following happens:

# buttersink /mnt/storage/backup/ ssh://remotebackup@hermes/mnt/backup/
  Remote version: {u'btrfs': u'Btrfs v3.18.2', u'buttersink': u'0.6-2-g2ad4585', u'linux': u'Linux-3.16.0-0.bpo.4-amd64-x86_64-with-debian-7.8'}
  measured size (52.38 GiB), estimated size (52.38 GiB)
  Optimal synchronization:
  52.38 GiB from 2 diffs in btrfs /mnt/storage/backup
  52.38 GiB from 2 diffs in TOTAL
  Xfer: bd7d...26e1 /mnt/storage/backup/sx-20150322-010000-utc from None (52.01 GiB)
  btrfs send errorsnt 20 MiB of 52.01 GiB (0%) ( 169 Mbps) ETA: 0:43:59.528292
At subvol /mnt/storage/backup/sx-20150322-010000-utc
ERROR: failed to dump stream. Broken pipe

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/buttersink/buttersink.py", line 248, in main
    diff.sendTo(dest, chunkSize=args.part_size << 20)
  File "/usr/lib/python2.7/site-packages/buttersink/Store.py", line 344, in sendTo
    transfer(sendContext, receiveContext, chunkSize)
  File "/usr/lib/python2.7/site-packages/buttersink/Store.py", line 268, in transfer
    writer.write(data)
  File "/usr/lib/python2.7/site-packages/buttersink/SSHStore.py", line 155, in write
    raise Exception(result)
Exception: {u'errorType': u'AttributeError', u'command': u'write', u'error': u"'bytearray' object has no attribute 'tobytes'", u'server': True}

I updated btrfs-progs on the server since my last coment. The laptop runs Arch Linux and buttersink seems to work without problems.

So long

[AmesCornish]

Robin,

Thanks for the bug report. I've just pushed a patch to GitHub. Could you test and make sure it fixes your problem?

Thanks!

• Ames

On Tue, Mar 24, 2015 at 10:50 AM, Robin Nehls notifications@github.com wrote:

Hi, thanks for looking into this, but there seems to be another problem. running

buttersink /mn/btrfs/snapshots

on my debain wheezy server, now works and lists subvolumes. but when I try to sync snapshots from my laptop to my server the following happens:

buttersink /mnt/storage/backup/ ssh://remotebackup@hermes/mnt/backup/hyperion/

Remote version: {u'btrfs': u'Btrfs v3.18.2', u'buttersink': u'0.6-2-g2ad4585', u'linux': u'Linux-3.16.0-0.bpo.4-amd64-x86_64-with-debian-7.8'} measured size (52.38 GiB), estimated size (52.38 GiB) Optimal synchronization: 52.38 GiB from 2 diffs in btrfs /mnt/storage/backup 52.38 GiB from 2 diffs in TOTAL Xfer: bd7d...26e1 /mnt/storage/backup/sx-20150322-010000-utc from None (52.01 GiB) btrfs send errorsnt 20 MiB of 52.01 GiB (0%) ( 169 Mbps) ETA: 0:43:59.528292 At subvol /mnt/storage/backup/sx-20150322-010000-utc ERROR: failed to dump stream. Broken pipe

Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/buttersink/buttersink.py", line 248, in main diff.sendTo(dest, chunkSize=args.part_size << 20) File "/usr/lib/python2.7/site-packages/buttersink/Store.py", line 344, in sendTo transfer(sendContext, receiveContext, chunkSize) File "/usr/lib/python2.7/site-packages/buttersink/Store.py", line 268, in transfer writer.write(data) File "/usr/lib/python2.7/site-packages/buttersink/SSHStore.py", line 155, in write raise Exception(result) Exception: {u'errorType': u'AttributeError', u'command': u'write', u'error': u"'bytearray' object has no attribute 'tobytes'", u'server': True}

I updated btrfs-progs on the server since my last coment. The laptop runs Arch Linux and buttersink seems to work without problems.

So long

— Reply to this email directly or view it on GitHub https://github.com/AmesCornish/buttersink/issues/2#issuecomment-85619096 .

[man0lis]

Hi, now everything seems to work perfectly. Thank you very much. I was able to sync my backup snapshots to my server and my new fancy backup setup is now complete

So long