Issues building 1.0.0 release

[gregswift]

@thedrow I'm working through updating the fedora package, and during the builds i'm seeing some issues. The latest one looks like a circular dependency.

+ /usr/bin/python2 setup.py build '--executable=/usr/bin/python2 -s'
Traceback (most recent call last):
  File "setup.py", line 23, in <module>
    url = "http://augeas.net/",
  File "/usr/lib64/python2.7/distutils/core.py", line 111, in setup
    _setup_distribution = dist = klass(attrs)
  File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 320, in __init__
    _Distribution.__init__(self, attrs)
  File "/usr/lib64/python2.7/distutils/dist.py", line 287, in __init__
    self.finalize_options()
  File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 387, in finalize_options
    ep.load()(self, ep.name, value)
  File "/usr/lib64/python2.7/site-packages/cffi/setuptools_ext.py", line 187, in cffi_modules
    add_cffi_module(dist, cffi_module)
  File "/usr/lib64/python2.7/site-packages/cffi/setuptools_ext.py", line 49, in add_cffi_module
    execfile(build_file_name, mod_vars)
  File "/usr/lib64/python2.7/site-packages/cffi/setuptools_ext.py", line 25, in execfile
    exec(code, glob, glob)
  File "augeas/ffi.py", line 44, in <module>
    lib = ffi.dlopen("augeas")
  File "/usr/lib64/python2.7/site-packages/cffi/api.py", line 139, in dlopen
    lib, function_cache = _make_ffi_library(self, name, flags)
  File "/usr/lib64/python2.7/site-packages/cffi/api.py", line 769, in _make_ffi_library
    backendlib = _load_backend_lib(backend, libname, flags)
  File "/usr/lib64/python2.7/site-packages/cffi/api.py", line 757, in _load_backend_lib
    raise OSError("library not found: %r" % (name,))
OSError: library not found: 'augeas'

[gregswift]

Added augeas-devel to BuildRequires and got past that. Now I get

/usr/include/augeas.h:24:10: fatal error: libxml/tree.h: No such file or directory
 #include <libxml/tree.h>
          ^~~~~~~~~~~~~~~
compilation terminated.
error: command 'gcc' failed with exit status 1
error: Bad exit status from /var/tmp/rpm-tmp.zinlr4 (%build)
    Bad exit status from /var/tmp/rpm-tmp.zinlr4 (%build)

And despite having added libxml2-devel, which provides this file, i still get this error.

[ohemorange]

My error is slightly different, before installing any new os packages:

  Running setup.py bdist_wheel for python-augeas: started
  Running setup.py bdist_wheel for python-augeas: finished with status 'error'
  Complete output from command /home/erica/certbot/.tox/py27/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-Bfip2b/python-augeas/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmp74ADFbpip-wheel- --python-tag cp27:
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-2.7
  creating build/lib.linux-x86_64-2.7/augeas
  copying augeas/ffi.py -> build/lib.linux-x86_64-2.7/augeas
  copying augeas/__init__.py -> build/lib.linux-x86_64-2.7/augeas
  running build_ext
  generating cffi module 'build/temp.linux-x86_64-2.7/augeas.c'
  creating build/temp.linux-x86_64-2.7
  building 'augeas' extension
  creating build/temp.linux-x86_64-2.7/build
  creating build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/augeas.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/augeas.o
  build/temp.linux-x86_64-2.7/augeas.c:434:35: fatal error: augeas.h: No such file or directory
                  #include <augeas.h>
                                     ^
  compilation terminated.
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for python-augeas
  Running setup.py clean for python-augeas
Failed to build python-augeas
Installing collected packages: python-augeas, certbot-apache
  Running setup.py install for python-augeas: started
    Running setup.py install for python-augeas: finished with status 'error'
    Complete output from command /home/erica/certbot/.tox/py27/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-Bfip2b/python-augeas/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-AaZEp1-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/erica/certbot/.tox/py27/include/site/python2.7/python-augeas:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/augeas
    copying augeas/ffi.py -> build/lib.linux-x86_64-2.7/augeas
    copying augeas/__init__.py -> build/lib.linux-x86_64-2.7/augeas
    running build_ext
    generating cffi module 'build/temp.linux-x86_64-2.7/augeas.c'
    creating build/temp.linux-x86_64-2.7
    building 'augeas' extension
    creating build/temp.linux-x86_64-2.7/build
    creating build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/augeas.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/augeas.o
    build/temp.linux-x86_64-2.7/augeas.c:434:35: fatal error: augeas.h: No such file or directory
                    #include <augeas.h>
                                       ^
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
Command "/home/erica/certbot/.tox/py27/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-Bfip2b/python-augeas/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-AaZEp1-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/erica/certbot/.tox/py27/include/site/python2.7/python-augeas" failed with error code 1 in /tmp/pip-build-Bfip2b/python-augeas/
ERROR: InvocationError: '/home/erica/certbot/.tox/py27/bin/pip install -e certbot-apache'

[gregswift]

Without any libxml related dependencies:

gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -fPIC -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/augeas.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/augeas.o
In file included from build/temp.linux-x86_64-2.7/augeas.c:434:0:
/usr/include/augeas.h:24:10: fatal error: libxml/tree.h: No such file or directory
 #include <libxml/tree.h>
          ^~~~~~~~~~~~~~~
compilation terminated.
error: command 'gcc' failed with exit status 1

With libxml added

gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -fPIC -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/augeas.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/augeas.o
In file included from build/temp.linux-x86_64-2.7/augeas.c:434:0:
/usr/include/augeas.h:24:10: fatal error: libxml/tree.h: No such file or directory
 #include <libxml/tree.h>
          ^~~~~~~~~~~~~~~
compilation terminated.
error: command 'gcc' failed with exit status 1
error: Bad exit status from /var/tmp/rpm-tmp.LJMtfN (%build)
    Bad exit status from /var/tmp/rpm-tmp.LJMtfN (%build)

so.. ya .. ??

[gregswift]

Following a stackoverflow post it appears to be a library issue. I'm a touch concerned about how that plays out but i was able to do a src build with:

C_INCLUDE_PATH=/usr/include/libxml2/ make build

@lutter put together a possible patch in #22 to fix this, but we both agree it might not be the right fix.

@thedrow since 1.0.0 is already released we are breaking ppl so we need to get this figured out.

[lutter]

@thedrow the 'build' target in Makefile is also broken as the command needs to be on its own line in a Makefile. make check doesn't work either, and specifying a PREFIX doesn't seem to install things in the prefix

[thedrow]

We can vendor both augeas and libxml2 in this library if required.

[gregswift]

What do you mean by vendor ? We should not be bundling anything.

[thedrow]

By vendor I mean include the libraries in the distribution and compile against them.

[rackergs]

thats what i thought you meant but wanted to make sure. so in Fedora we'd have to get an exemption allowing us to bundle, and i don't think they'd approve this since it just sounds like we need to figure out the correct way to build it. not sure about pypi's policy on this.

[thedrow]

You can bundle anything you want on PyPi. Gotcha so we'll have to find a proper fix.

[thedrow]

I'm going to keep this issue open until we find a proper fix for the problem.

[ohemorange]

Hi, does 1.0.1 have the vendorized libraries? The error message is unchanged for me even pulling down that latest version.

[ohemorange]

Hi again! Actually, we're coming up with an issue where some people with very old versions of our wrapper script always pull down the latest version of python-augeas off of pip. Until the issues are fixed, is there any chance 0.5.0 could be reuploaded as 1.0.2? People's certificates will fail to renew until this is fixed, meaning that their websites will be insecure.

We've pinned 0.5.0 in more recent and self-upgrading versions, but you know how upgrading goes.

[thedrow]

@ohemorange Which distribution are you on? Is libxml2 installed? 1.0.1 does not have any of the dependencies vendorized.

[ohemorange]

Ubuntu 16.04, yes: ii libxml2:amd64 2.9.3+dfsg1-1ubuntu amd64 GNOME XML library

[ohemorange]

But, I should mention, it's not just me who's hitting this issue. Our users who set things up a while ago to run automatically are having to go in and manually change things because of the broken package:

https://community.letsencrypt.org/t/certbot-auto-upgrades-corrupt-certbot-and-not-able-to-renew/30889/

[lutter]

@thedrow seeing how thoroughly this breaks users, and how you haven't found a proper fix for this, I think reverting this and releasing as 1.0.2 makes the most sense. I don't think it's fair to let's encrypt that they have to bear the burden of this breakage - this has been going on for over a week, and we really need to get users back to something that works at least as well as what they had before.

Also, vendoring anything is absolutely the wrong approach to solving this. The previous version had no issues with building, and we need to get back to that world. My suggestion is to use pkg-config to find the right include directories and parametrize the build accordingly.

[yan12125]

I guess on Debian-based systems (Ubuntu, Mint, etc.), libxml2-dev is also necessary besides libxml2.

[thedrow]

@lutter I think the real problem is that now python-augeas requires a dependency it didn't require before. I figured that marking the new release as 1.0 is enough to reflect this. Turns out I was wrong.

[thedrow]

I just released 1.0.2 which fixes this issue.