search

g2p / bedup

Btrfs deduplication
http://pypi.python.org/pypi/bedup
GNU General Public License v2.0
322 stars 50 forks source link

syncfs backport #10

Closed ClashTheBunny closed 11 years ago

ClashTheBunny commented 11 years ago

Original title: Undefined symbol: syncfs on Debian Wheezy (testing) ARM

I get this error when I try to build. I also tried with the latest btrfs-tools source in btrfs-tools. I'm working on a fully updated Debian Wheezy install on my ARM NAS. I can't find any references to this syncfs symbol in any place that I wouldn't have installed. I am running on my own kernel 3.8.0-rc5 compiled on the ARM box with headers and all installed. I also tried apt-get build-dep btrfs-tools and building the latest btrfs-tools from source.

Linux <hostname> 3.8.0-rc5 #1 Tue Feb 5 01:57:02 EET 2013 armv5tel GNU/Linux btrfs-tools package version 0.19+20120328-7

cffi is 0.4.2 and I'm on bedup's master branch.

~/bedup$ python setup.py install --user
Traceback (most recent call last):
  File "setup.py", line 6, in <module>
    from bedup.platform import get_ext_modules
  File "/home/rmason/bedup/bedup/platform/__init__.py", line 21, in <module>
    from . import (
  File "/home/rmason/bedup/bedup/platform/syncfs.py", line 31, in <module>
    ext_package='bedup')
  File "/usr/local/lib/python2.7/dist-packages/cffi/api.py", line 284, in verify
    lib = self.verifier.load_library()
  File "/usr/local/lib/python2.7/dist-packages/cffi/verifier.py", line 62, in load_library
    return self._load_library()
  File "/usr/local/lib/python2.7/dist-packages/cffi/verifier.py", line 142, in _load_library
    return self._vengine.load_library()
  File "/usr/local/lib/python2.7/dist-packages/cffi/vengine_cpy.py", line 123, in load_library
    raise ffiplatform.VerificationError(error)
cffi.ffiplatform.VerificationError: importing '/home/rmason/bedup/bedup/platform/__pycache__/_cffi__x83ddfcbex3fe6f2b1.so': /home/rmason/bedup/bedup/platform/__pycache__/_cffi__x83ddfcbex3fe6f2b1.so: undefined symbol: syncfs

If I pip install bedup, I get this pip.log telling me essentially the same thing:

Downloading/unpacking bedup

  Getting page http://pypi.python.org/simple/bedup
  URLs to search for versions for bedup:
  * http://pypi.python.org/simple/bedup/
  Getting page https://github.com/g2p/bedup
  Analyzing links from page http://pypi.python.org/simple/bedup/
    Found link http://pypi.python.org/packages/source/b/bedup/bedup-0.0.7.tar.gz#md5=0ad30f2b65f367eea717dfa45da6bc90 (from http://pypi.python.org/simple/bedup/), version: 0.0.7
    Found link http://pypi.python.org/packages/source/b/bedup/bedup-0.0.2.tar.gz#md5=9400593c692361f907816ea913eeda95 (from http://pypi.python.org/simple/bedup/), version: 0.0.2
    Found link http://pypi.python.org/packages/source/b/bedup/bedup-0.0.4.tar.gz#md5=89a1e9e83fc98e83f4a1f3620e81ee20 (from http://pypi.python.org/simple/bedup/), version: 0.0.4
    Found link http://pypi.python.org/packages/source/b/bedup/bedup-0.0.6.tar.gz#md5=9bb525ebfc3560537632a8f370195a57 (from http://pypi.python.org/simple/bedup/), version: 0.0.6
    Found link http://pypi.python.org/packages/source/b/bedup/bedup-0.0.1.tar.gz#md5=8bba6ff035bc0ce7d446cf44a4c4e6e3 (from http://pypi.python.org/simple/bedup/), version: 0.0.1
    Found link http://pypi.python.org/packages/source/b/bedup/bedup-0.0.8.tar.gz#md5=4c9ecd22d933d5988060ac805a3060e1 (from http://pypi.python.org/simple/bedup/), version: 0.0.8
    Found link http://pypi.python.org/packages/source/b/bedup/bedup-0.0.5.tar.gz#md5=c64ed5e356539dd92963dd3c0cf83a75 (from http://pypi.python.org/simple/bedup/), version: 0.0.5
    Found link http://pypi.python.org/packages/source/b/bedup/bedup-0.0.3.tar.gz#md5=856046fc3238024ba97e93d5e0c9603c (from http://pypi.python.org/simple/bedup/), version: 0.0.3
    Skipping link https://github.com/g2p/bedup (from http://pypi.python.org/simple/bedup/); not a file
    Skipping link https://github.com/g2p/bedup#readme (from http://pypi.python.org/simple/bedup/); not a file
  Analyzing links from page https://github.com/g2p/bedup
    Skipping link https://github.com/opensearch.xml (from https://github.com/g2p/bedup); unknown archive format: .xml
    Skipping link https://github.com/fluidicon.png (from https://github.com/g2p/bedup); unknown archive format: .png
    Skipping link https://github.com/g2p/apple-touch-icon-114.png (from https://github.com/g2p/bedup); unknown archive format: .png
    Skipping link https://github.com/g2p/apple-touch-icon-144.png (from https://github.com/g2p/bedup); unknown archive format: .png
    Skipping link http://github-media-downloads.s3.amazonaws.com/github-logo.svg (from https://github.com/g2p/bedup); unknown archive format: .svg
    Skipping link https://github.com/favicon.ico (from https://github.com/g2p/bedup); unknown archive format: .ico
    Skipping link https://a248.e.akamai.net/assets.github.com/assets/github-2edf6f58a5335626270ffd1443694b5b6cf35623.css (from https://github.com/g2p/bedup); unknown archive format: .css
    Skipping link https://a248.e.akamai.net/assets.github.com/assets/github2-2f6b52ce3884d9a32c2dc1fd41266c20de7aa966.css (from https://github.com/g2p/bedup); unknown archive format: .css
    Skipping link https://github.com/g2p/bedup/tree/875a3a42cc62204999aa723024805d401490ae5a (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/commits/master.atom (from https://github.com/g2p/bedup); unknown archive format: .atom
    Skipping link https://github.com/ (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/explore (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/search (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/features (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/blog (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/signup (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/login?return_to=%2Fg2p%2Fbedup (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/stargazers (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/network (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/pulls (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/issues (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/graphs (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/languages/Python (from https://github.com/g2p/bedup); not a file
    Skipping link http://pypi.python.org/pypi/bedup (from https://github.com/g2p/bedup); not a file
    Could not parse version from link: https://github.com/g2p/bedup/archive/master.zip (from https://github.com/g2p/bedup)
    Skipping link https://github.com/g2p/bedup/archive/master.zip (from https://github.com/g2p/bedup); wrong project name (not bedup)
    Skipping link https://github.com/g2p/bedup.git (from https://github.com/g2p/bedup); unknown archive format: .git
    Skipping link https://github.com/g2p/git@github.com:g2p/bedup.git (from https://github.com/g2p/bedup); unknown archive format: .git
    Skipping link git://github.com/g2p/bedup.git (from https://github.com/g2p/bedup); unknown archive format: .git
    Skipping link https://github.com/g2p/bedup/tags (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/tree/master (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/tree/travis-unicode-bug (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/tree/troubleshooting/cffi-py3.3 (from https://github.com/g2p/bedup); unknown archive format: .3
    Skipping link https://github.com/g2p/bedup/tree/troubleshooting/comm1-gotit (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/tree/troubleshooting/comm1-minified (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/tree/wip/commit-in-thread (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/tree/v0.0.8 (from https://github.com/g2p/bedup); unknown archive format: .8
    Skipping link https://github.com/g2p/bedup/tree/v0.0.7 (from https://github.com/g2p/bedup); unknown archive format: .7
    Skipping link https://github.com/g2p/bedup/tree/v0.0.6 (from https://github.com/g2p/bedup); unknown archive format: .6
    Skipping link https://github.com/g2p/bedup/tree/v0.0.5 (from https://github.com/g2p/bedup); unknown archive format: .5
    Skipping link https://github.com/g2p/bedup/tree/v0.0.4 (from https://github.com/g2p/bedup); unknown archive format: .4
    Skipping link https://github.com/g2p/bedup/tree/v0.0.3 (from https://github.com/g2p/bedup); unknown archive format: .3
    Skipping link https://github.com/g2p/bedup/tree/v0.0.2 (from https://github.com/g2p/bedup); unknown archive format: .2
    Skipping link https://github.com/g2p/bedup/tree/v0.0.1 (from https://github.com/g2p/bedup); unknown archive format: .1
    Skipping link https://github.com/g2p/bedup/commits/master (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/branches (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/find/master (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/tree/master/bedup (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/blob/master/.gitignore (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/blob/master/.gitmodules (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/blob/master/.travis.yml (from https://github.com/g2p/bedup); unknown archive format: .yml
    Skipping link https://github.com/g2p/bedup/blob/master/COPYING (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup/blob/master/MANIFEST.in (from https://github.com/g2p/bedup); unknown archive format: .in
    Skipping link https://github.com/g2p/bedup/blob/master/README.rst (from https://github.com/g2p/bedup); unknown archive format: .rst
    Skipping link https://github.com/g2p/bedup/blob/master/setup.py (from https://github.com/g2p/bedup); unknown archive format: .py
    Skipping link https://github.com/g2p/bedup/blob/master/tox.ini (from https://github.com/g2p/bedup); unknown archive format: .ini
    Skipping link https://github.com/g2p/bedup#requirements (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup#installation (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup#option-1-recommended-from-a-git-clone (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup#option-2-from-a-pypi-release (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup#running (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup#hacking (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup#implementation (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup#locking (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup#subvolumes (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/g2p/bedup#build-status (from https://github.com/g2p/bedup); not a file
    Skipping link https://travis-ci.org/g2p/bedup (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/about (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/contact (from https://github.com/g2p/bedup); not a file
    Skipping link http://enterprise.github.com/ (from https://github.com/g2p/bedup); not a file
    Skipping link http://status.github.com/ (from https://github.com/g2p/bedup); not a file
    Skipping link http://mac.github.com/ (from https://github.com/g2p/bedup); not a file
    Skipping link http://windows.github.com/ (from https://github.com/g2p/bedup); not a file
    Skipping link http://eclipse.github.com/ (from https://github.com/g2p/bedup); not a file
    Skipping link http://mobile.github.com/ (from https://github.com/g2p/bedup); not a file
    Skipping link http://get.gaug.es/ (from https://github.com/g2p/bedup); not a file
    Skipping link http://speakerdeck.com (from https://github.com/g2p/bedup); not a file
    Skipping link https://gist.github.com (from https://github.com/g2p/bedup); not a file
    Skipping link http://jobs.github.com/ (from https://github.com/g2p/bedup); not a file
    Skipping link http://help.github.com/ (from https://github.com/g2p/bedup); not a file
    Skipping link http://developer.github.com/ (from https://github.com/g2p/bedup); not a file
    Skipping link http://github.github.com/github-flavored-markdown/ (from https://github.com/g2p/bedup); not a file
    Skipping link http://pages.github.com/ (from https://github.com/g2p/bedup); not a file
    Skipping link http://training.github.com/ (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/edu (from https://github.com/g2p/bedup); not a file
    Skipping link http://shop.github.com (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/plans (from https://github.com/g2p/bedup); not a file
    Skipping link http://octodex.github.com/ (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/site/terms (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/site/privacy (from https://github.com/g2p/bedup); not a file
    Skipping link https://github.com/security (from https://github.com/g2p/bedup); not a file
  Using version 0.0.8 (newest of versions: 0.0.8, 0.0.7, 0.0.6, 0.0.5, 0.0.4, 0.0.3, 0.0.2, 0.0.1)
  Downloading from URL http://pypi.python.org/packages/source/b/bedup/bedup-0.0.8.tar.gz#md5=4c9ecd22d933d5988060ac805a3060e1 (from http://pypi.python.org/simple/bedup/)
  Running setup.py egg_info for package bedup

    bedup/platform/__pycache__/_cffi__x83ddfcbex3fe6f2b1.c: In function ‘_cffi_f_syncfs’:

    bedup/platform/__pycache__/_cffi__x83ddfcbex3fe6f2b1.c:166:3: warning: implicit declaration of function ‘syncfs’ [-Wimplicit-function-declaration]

    Traceback (most recent call last):

      File "<string>", line 14, in <module>

      File "/home/rmason/bedup/build/bedup/setup.py", line 6, in <module>

        from bedup.platform import get_ext_modules

      File "bedup/platform/__init__.py", line 21, in <module>

        from . import (

      File "bedup/platform/syncfs.py", line 31, in <module>

        ext_package='bedup')

      File "/usr/local/lib/python2.7/dist-packages/cffi/api.py", line 284, in verify

        lib = self.verifier.load_library()

      File "/usr/local/lib/python2.7/dist-packages/cffi/verifier.py", line 62, in load_library

        return self._load_library()

      File "/usr/local/lib/python2.7/dist-packages/cffi/verifier.py", line 142, in _load_library

        return self._vengine.load_library()

      File "/usr/local/lib/python2.7/dist-packages/cffi/vengine_cpy.py", line 123, in load_library

        raise ffiplatform.VerificationError(error)

    cffi.ffiplatform.VerificationError: importing '/home/rmason/bedup/build/bedup/bedup/platform/__pycache__/_cffi__x83ddfcbex3fe6f2b1.so': /home/rmason/bedup/build/bedup/bedup/platform/__pycache__/_cffi__x83ddfcbex3fe6f2b1.so: undefined symbol: syncfs

    Complete output from command python setup.py egg_info:

    bedup/platform/__pycache__/_cffi__x83ddfcbex3fe6f2b1.c: In function ‘_cffi_f_syncfs’:

bedup/platform/__pycache__/_cffi__x83ddfcbex3fe6f2b1.c:166:3: warning: implicit declaration of function ‘syncfs’ [-Wimplicit-function-declaration]

Traceback (most recent call last):

  File "<string>", line 14, in <module>

  File "/home/rmason/bedup/build/bedup/setup.py", line 6, in <module>

    from bedup.platform import get_ext_modules

  File "bedup/platform/__init__.py", line 21, in <module>

    from . import (

  File "bedup/platform/syncfs.py", line 31, in <module>

    ext_package='bedup')

  File "/usr/local/lib/python2.7/dist-packages/cffi/api.py", line 284, in verify

    lib = self.verifier.load_library()

  File "/usr/local/lib/python2.7/dist-packages/cffi/verifier.py", line 62, in load_library

    return self._load_library()

  File "/usr/local/lib/python2.7/dist-packages/cffi/verifier.py", line 142, in _load_library

    return self._vengine.load_library()

  File "/usr/local/lib/python2.7/dist-packages/cffi/vengine_cpy.py", line 123, in load_library

    raise ffiplatform.VerificationError(error)

cffi.ffiplatform.VerificationError: importing '/home/rmason/bedup/build/bedup/bedup/platform/__pycache__/_cffi__x83ddfcbex3fe6f2b1.so': /home/rmason/bedup/build/bedup/bedup/platform/__pycache__/_cffi__x83ddfcbex3fe6f2b1.so: undefined symbol: syncfs

----------------------------------------

Command python setup.py egg_info failed with error code 1 in /home/rmason/bedup/build/bedup

Exception information:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 104, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 245, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1009, in prepare_files
    req_to_install.run_egg_info()
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 225, in run_egg_info
    command_desc='python setup.py egg_info')
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 256, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command python setup.py egg_info failed with error code 1 in /home/rmason/bedup/build/bedup
g2p commented 11 years ago

syncfs is in glibc 2.14. You can build on an older, unsupported glibc by commenting out the int syncfs(int fd) line in bedup/platform/syncfs.py; the --flush flag uses syncfs and will cause a runtime error after that.

g2p commented 11 years ago

I've backported syncfs for older glibc versions. Please test, and tell me if there are any further compatibility problems on Wheezy.

g2p commented 11 years ago

I've tested on Wheezy, closing again.

ClashTheBunny commented 11 years ago

Yep, I would also like to confirm that this fixed my problem. Thanks so much for the fast response!