dm.xmlsec.binding broken?

navneil commented 9 years ago

I can't get dm.xmlsec.binding to work on my fedora host. It just core dumps. Has anyone gotten this to work. As far as I can tell, I have everything installed cleanly. I find dm.xmlsec.binding to be very unreliable and modified python-saml to use the xmlsec1 binary instead.

pitbulk commented 9 years ago

@navneil , Why do you say dm.xmlsec.binding is unreliable?

dm.xmlsec.binding is used in dm.zope.saml2. After review many libraries, I choosedm.xmlsec.binding. It has documentation https://pypi.python.org/pypi/dm.xmlsec.binding/1.3.1 and is maintained.

What would you propose?

navneil commented 9 years ago

I can't get dm.xmlsec.binding to work on different linux distributions. Even after installing cleanly, it core dumps on import dm.xmlsec.binding. What distribution are you using?

Also, have you thought of using the xmlsec1 binaries instead of the python bindings.

pitbulk commented 9 years ago

I tested it on Redhat and Ubuntu. I dislike the use of xmlsec1 binaries execution. How are you installing the python-saml library? Do you use pip?

Can you take a look on this Travis trace? https://travis-ci.org/onelogin/python-saml As you see is installed as expected.

Also, take a look on the installation section of: https://pypi.python.org/pypi/dm.xmlsec.binding/1.3.1

What you get after those commands?

sudo yum install python-setuptools python-setuptools-devel python-pip
pip install python-saml

cjsteinke commented 9 years ago

Hi, I found this during a google search and I am having the same problem. I get a segmentation fault when I try to load the module as well.

This is on RHEL6.6, I followed the instructions from the Travis trace (with modifications for RHEL) and ensured all the necessary packages are installed.

In order to get past the M2Crypto failures I needed to download the package from the python web site and then apply the fedora_setup from the git clone of M2Crypto (which is an older version than the one available from python pypi)because it's NOT included with the download from pypi.

I also needed to install, which isn't mentioned anywhere, xmlsec1-openssl, xmlsec1-openssl-devel, libtool-ltdl and libtool-ltdl-devel (added the -devel packages to be on the safe side) Thi

Here is output from the coverage run

root@gandalf # coverage run --source=src/onelogin/saml2 --rcfile=tests/coverage.rc setup.py test
running test
running egg_info
writing requirements to src/python_saml.egg-info/requires.txt
writing src/python_saml.egg-info/PKG-INFO
writing top-level names to src/python_saml.egg-info/top_level.txt
writing dependency_links to src/python_saml.egg-info/dependency_links.txt
reading manifest file 'src/python_saml.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '__pycache__' found under directory '*'
warning: no previously-included files matching '*.py[co]' found under directory '*'
writing manifest file 'src/python_saml.egg-info/SOURCES.txt'
running build_ext
Segmentation fault (core dumped)

Here is my GDB run:

root@gandalf # gdb python2.7
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-75.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /opt/rh/python27/root/usr/bin/python2.7...(no debugging symbols found)...done.
Missing separate debuginfos, use: debuginfo-install python27-python-2.7.5-10.el6.x86_64
(gdb) run
Starting program: /opt/rh/python27/root/usr/bin/python2.7 
[Thread debugging using libthread_db enabled]
Python 2.7.5 (default, Dec  3 2013, 08:35:16) 
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import dm.xmlsec.binding

Program received signal SIGSEGV, Segmentation fault.
__pyx_f_4lxml_5etree_funicode (__pyx_v_s=0x10 <Address 0x10 out of bounds>) at src/lxml/lxml.etree.c:26833
26833   src/lxml/lxml.etree.c: No such file or directory.
    in src/lxml/lxml.etree.c

Any thoughts?

Thanks for any help.

cjsteinke commented 9 years ago

For shiz and giggles I tried on Fedora Core 20 and received the same results.

[root@sauron python-saml]#  coverage run --source=src/onelogin/saml2 --rcfile=tests/coverage.rc setup.py test
running test
running egg_info
writing requirements to src/python_saml.egg-info/requires.txt
writing src/python_saml.egg-info/PKG-INFO
writing top-level names to src/python_saml.egg-info/top_level.txt
writing dependency_links to src/python_saml.egg-info/dependency_links.txt
reading manifest file 'src/python_saml.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '__pycache__' found under directory '*'
warning: no previously-included files matching '*.py[co]' found under directory '*'
writing manifest file 'src/python_saml.egg-info/SOURCES.txt'
running build_ext
Segmentation fault

pitbulk commented 9 years ago

Hi @cjsteinke and @navneil , Sorry to see that you experiencing problems installing python-saml

@cjsteinke , I install M2Crypto using pypi: https://pypi.python.org/pypi/M2Crypto (openssl and swig are dependences from this library). You can read in the documentation of python-saml, at https://github.com/onelogin/python-saml#dependences you can see libxmlsec1-dev, and libxmlsec1-openssl is a dependece of this package.

I googled the error and it seems a bug with the lxml library (lxml is a dependece of dm.xmlsec.binding): https://bugs.launchpad.net/lxml/+bug/928333

pitbulk commented 9 years ago

Hi @navneil and @cjsteinke , sorry to see that python-saml is not working on Fedora.

In python-saml I install M2Crypto using pypi (https://pypi.python.org/pypi/M2Crypto). This library requires openssl and swig.

I googled the error and it seems a bug on the lxml library (a dependence of dm.xmlsec.binding)

pitbulk commented 9 years ago

I asked a friend to try get it working on Fedora.

@cjsteinke can you try to use the lxml library? Maybe an issue that this library was not compiled correctly. Did you install the python-dev, libxml2-dev and libxslt-dev libraries?

In Ubuntu, when building 'dm.xmlsec.binding' I see:

    building 'dm.xmlsec.binding._xmlsec' extension
    gcc -pthread -fno-strict-aliasing -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_NO_SIZE_T=1 -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_OPENSSL_098=1 -DXMLSEC_CRYPTO_OPENSSL=1 -DXMLSEC_CRYPTO=openssl -I/home/travis/virtualenv/python2.7.8/build/dm.xmlsec.binding/lxml-3.4.0-py2.7-linux-x86_64.egg/lxml -I/home/travis/virtualenv/python2.7.8/build/dm.xmlsec.binding/lxml-3.4.0-py2.7-linux-x86_64.egg/lxml/includes -Isrc -I/usr/include/libxml2 -I/usr/include/xmlsec1 -I/opt/python/2.7.8/include/python2.7 -c src/_xmlsec.c -o build/temp.linux-x86_64-2.7/src/_xmlsec.o
    gcc -pthread -shared -L/opt/python/2.7.8/lib -Wl,-rpath=/opt/python/2.7.8/lib build/temp.linux-x86_64-2.7/src/_xmlsec.o -L/usr/lib -L/usr/lib/x86_64-linux-gnu -L/opt/python/2.7.8/lib -lxml2 -lxmlsec1-openssl -lxmlsec1 -lxslt -lssl -lcrypto -lpython2.7 -o build/lib.linux-x86_64-2.7/dm/xmlsec/binding/_xmlsec.so

pitbulk commented 9 years ago

Can you try to execute this in order to know the versión installed:

import sys
from lxml import etree

print("%-20s: %s" % ('Python', sys.version_info))
print("%-20s: %s" % ('lxml.etree', etree.LXML_VERSION))
print("%-20s: %s" % ('libxml used', etree.LIBXML_VERSION))
print("%-20s: %s" % ('libxml compiled', etree.LIBXML_COMPILED_VERSION))
print("%-20s: %s" % ('libxslt used', etree.LIBXSLT_VERSION))
print("%-20s: %s" % ('libxslt compiled', etree.LIBXSLT_COMPILED_VERSION))

Also in the dm.xmlsec.binding documentation appears:

This package interfaces with lxml via its Cython interface (described in etreepublic.pxd).
Some operating system installations for lxml lack the respective files. In those cases,
you may need to download an lxml source distribution and let the environment variable
LXML_HOME point to its root.

don't know if maybe is missing this LXML_HOME path.

Have you tried to install lxml using pypi? https://pypi.python.org/pypi/lxml/3.4.0

navneil commented 9 years ago

I got the normal source but I still get dm.xmlsec.binding to crash:

Python 2.7.5 (default, Dec 3 2013, 08:35:16) [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information.

import dm.xmlsec.binding

Program received signal SIGSEGV, Segmentation fault. pyx_f_4lxml_5etree_funicode (pyx_v_s=0x10 <Address 0x10 out of bounds>) at src/lxml/lxml.etree.c:26833 26833 if (!__pyx_t_1) break;

This is what I am using:

Python : sys.version_info(major=2, minor=7, micro=5, releaselevel='final', serial=0) lxml.etree : (3, 4, 0, 0) libxml used : (2, 7, 6) libxml compiled : (2, 7, 6) libxslt used : (1, 1, 26) libxslt compiled : (1, 1, 26)

On Wed, Oct 29, 2014 at 1:55 AM, Sixto Martin notifications@github.com wrote:

In /src/lxml there are only .pyx files, no .c

So you need to get "normal release sources" to have the C files pre-compiled, e.g. from http://pypi.python.org/pypi/lxml. If you check out from git, you need Cython.

http://stackoverflow.com/questions/14588404/build-lxml-pre-compiled-c-cython-files-missing-solved

— Reply to this email directly or view it on GitHub https://github.com/onelogin/python-saml/issues/30#issuecomment-60890811.

pitbulk commented 9 years ago

I sent a mail to the author of dm.xmlsec.binding (Dieter). He was very kind helping me with some doubts with the library in the past. I hope he has a solution for this problem.

Sorry for the inconveniences.

cjsteinke commented 9 years ago

Output:

Python 2.7.5 (default, Dec  3 2013, 08:35:16) 
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> from lxml import etree
>>> print("%-20s: %s" % ('Python', sys.version_info))
Python              : sys.version_info(major=2, minor=7, micro=5, releaselevel='final', serial=0)
>>> print("%-20s: %s" % ('lxml.etree', etree.LXML_VERSION))
lxml.etree          : (3, 4, 0, 0)
>>> print("%-20s: %s" % ('libxml used', etree.LIBXML_VERSION))
libxml used         : (2, 7, 6)
>>> print("%-20s: %s" % ('libxml compiled', etree.LIBXML_COMPILED_VERSION))
libxml compiled     : (2, 7, 6)
>>> print("%-20s: %s" % ('libxslt used', etree.LIBXSLT_VERSION))
libxslt used        : (1, 1, 26)
>>> print("%-20s: %s" % ('libxslt compiled', etree.LIBXSLT_COMPILED_VERSION))
libxslt compiled    : (1, 1, 26)

I was able to upgrade lxml, I had 2.2.3 installed and it upgraded it to 3.4.0.

I also downloaded lxml and pointed LXML_HOME to it's root and no luck.

Thanks for your help.

pitbulk commented 9 years ago

I found this resource: http://nz-realme.readthedocs.org/en/latest/installation.html#libxml2-and-xmldsig Can you try install that libxml2 as suggested?

navneil commented 9 years ago

I finally got dm.xmlsec.binding to work on Fedora but I needed to comment out this line

transformByHref = dict((t.href, t) for t in transforms)

in file dm/xmlsec/binding/init.py

I don't know what this line does but I am able to run the python-saml toolkit and was able to get the demo-flask app to work.

pitbulk commented 9 years ago

Nice, I mailed to Dieter that info. Let's see if he can fix that issue.

Thanks for getting involved!

cjsteinke commented 9 years ago

I confirmed that I installed libxml2,but same story.

Not sure if this will help, I ran a GDB session in the lxml-3.4.0 source tree

Contents of script to load dm.xmlsec.binding

cat ../onelogin/python-saml/loadxmlsec.py 
#!/usr/bin/env python2.7
import dm.xmlsec.binding

print "Foo!"

gdb python2.7
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-75.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /opt/rh/python27/root/usr/bin/python2.7...(no debugging symbols found)...done.
Missing separate debuginfos, use: debuginfo-install python27-python-2.7.5-10.el6.x86_64
(gdb) run ../onelogin/python-saml/loadxmlsec.py 
Starting program: /opt/rh/python27/root/usr/bin/python2.7 ../onelogin/python-saml/loadxmlsec.py
[Thread debugging using libthread_db enabled]

Program received signal SIGSEGV, Segmentation fault.
__pyx_f_4lxml_5etree_funicode (__pyx_v_s=0x10 <Address 0x10 out of bounds>) at src/lxml/lxml.etree.c:26833
26833       if (!__pyx_t_1) break;
(gdb) bt
#0  __pyx_f_4lxml_5etree_funicode (__pyx_v_s=0x10 <Address 0x10 out of bounds>) at src/lxml/lxml.etree.c:26833
#1  0x00007ffff123da2e in pyunicode (__pyx_v_s=<value optimized out>) at src/lxml/lxml.etree.c:179443
#2  0x00007ffff18061df in __pyx_f_2dm_6xmlsec_7binding_7_xmlsec_xmlChar2py (__pyx_v_xs=<value optimized out>) at src/_xmlsec.c:8352
#3  0x00007ffff18065b1 in __pyx_pf_2dm_6xmlsec_7binding_7_xmlsec_9Transform_4href___get__ (o=<value optimized out>, x=<value optimized out>) at src/_xmlsec.c:2421
#4  __pyx_pw_2dm_6xmlsec_7binding_7_xmlsec_9Transform_4href_1__get__ (o=<value optimized out>, x=<value optimized out>) at src/_xmlsec.c:2399
#5  __pyx_getprop_2dm_6xmlsec_7binding_7_xmlsec_9Transform_href (o=<value optimized out>, x=<value optimized out>) at src/_xmlsec.c:8651
#6  0x00007ffff7c98968 in ?? () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#7  0x00007ffff7cbfd1b in _PyObject_GenericGetAttrWithDict () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#8  0x00007ffff7d18948 in PyEval_EvalFrameEx () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#9  0x00007ffff7c9e337 in ?? () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#10 0x00007ffff7c7f7fb in PyIter_Next () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#11 0x00007ffff7cbb78f in PyDict_MergeFromSeq2 () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#12 0x00007ffff7cbc0a7 in ?? () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#13 0x00007ffff7cd74d8 in ?? () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#14 0x00007ffff7c80913 in PyObject_Call () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#15 0x00007ffff7d1c7c6 in PyEval_EvalFrameEx () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#16 0x00007ffff7d1ec4e in PyEval_EvalCodeEx () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#17 0x00007ffff7d1ed62 in PyEval_EvalCode () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#18 0x00007ffff7d2e522 in PyImport_ExecCodeModuleEx () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#19 0x00007ffff7d2e948 in ?? () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#20 0x00007ffff7d2f77a in ?? () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#21 0x00007ffff7d2fe11 in ?? () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#22 0x00007ffff7d30084 in ?? () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#23 0x00007ffff7d30f10 in PyImport_ImportModuleLevel () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#24 0x00007ffff7d1681f in ?? () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#25 0x00007ffff7c80913 in PyObject_Call () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#26 0x00007ffff7d16d03 in PyEval_CallObjectWithKeywords () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#27 0x00007ffff7d18b16 in PyEval_EvalFrameEx () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#28 0x00007ffff7d1ec4e in PyEval_EvalCodeEx () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#29 0x00007ffff7d1ed62 in PyEval_EvalCode () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#30 0x00007ffff7d3914c in ?? () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#31 0x00007ffff7d39220 in PyRun_FileExFlags () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#32 0x00007ffff7d3a7bf in PyRun_SimpleFileExFlags () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#33 0x00007ffff7d4c2b6 in Py_Main () from /opt/rh/python27/root/usr/lib64/libpython2.7.so.1.0
#34 0x0000003df9a1ed5d in __libc_start_main () from /lib64/libc.so.6
#35 0x0000000000400649 in _start ()
(gdb) info threads
* 1 Thread 0x7ffff7bff700 (LWP 1252)  __pyx_f_4lxml_5etree_funicode (__pyx_v_s=0x10 <Address 0x10 out of bounds>) at src/lxml/lxml.etree.c:26833
(gdb)

pitbulk commented 9 years ago

Dieter replied:

Looks as if the "libxmlsec1" version in this setup in not fully appropriate -- maybe, the version is too old (as this is the case for Ubuntu 10.04).

"dm.xmlsec.binding" is quite sensible to the correct version of "libxmlsec1".

"libxmlsec1" identifies transforms by so called "transform id"s.
Those ids as in fact pointers to structures with (among others) an "href" field.

In order to make a transform accessible by Python, "dm.xmlsec.binding" must provide a binding for the corresponding "transform id"s.
Especially, it must list all transform ids it wants to support.
I have created this list based on "xmlsec 1.2.18.

The problem you observe seems to indicate that one of the listed transforms is missing (I would expect a linking error in this case)
or its "href" field has an inappropriate value.

pitbulk commented 9 years ago

And after check the error trace:

This strenghens my assumption from the previous message.

As you can see, "dm.xmlsec.binding" tries to get the "href" from a transform
("__pyx_pf_2dm_6xmlsec_7binding_7_xmlsec_9Transform_4href___get__").
It gets some value (maybe "NULL", maybe "0x10") and hands it over to the "xmlChar2py" function of "dm.xmlsec.binding". This in turn
calls the "funicode" function from "lxml" which crashes due to a bad pointer ("0x10").

I interpret this as follows:
some of the transforms in the transform list used by "dm.xmlsec.binding" are not defined by the local "libxmlsec1" C library (version probably too old).
Rather than generating an error during the dynamic link phase, the linker returns an inappropriate value for the missing symbols
(likely "NULL"). Determining the "href" may lead to "0x10" which lets "lxml"'s "funicode" crash.

"dm.xmlsec.binding" generates its transform list and it might be possible to filter out transforms with inappropriate ids (e.g. "NULL" ids).
However, a too old version of "libxmlsec1" may have other consequences besides unsupported transforms. E.g. the "libxmlsec1" version
of Ubuntu 10.04 fails to support a function vital to "dm.xmlsec.binding", a function vital for "lxml" integration. Working to avoid the SIGSEGV
with the transforms would invariably cause another SIGSEGV later.
In this case, nothing would help other than an upgrade of "libxmlsec1".

cjsteinke commented 9 years ago

These are the version of xmlsec1 that have installed on my system:

xmlsec1-openssl-devel-1.2.20-4.el6.x86_64
xmlsec1-devel-1.2.20-4.el6.x86_64
xmlsec1-openssl-1.2.20-4.el6.x86_64
xmlsec1-1.2.20-4.el6.x86_64

So these version are newer than the 1.2.18 version used by Dieter.

I have noticed that packages from Ubuntu sometimes have bug fixes and enhancements that don't get pushed out to or included in other releases of the package for other distros and it's really annoying. In one particular case I was working with building a VPN server using PPTP and the Ubuntu version worked fine with IOS 6 using the same verison, off only by a subversion, but on RHEL or CENTOS it failed to work with IOS 6 devices, because there was a bug that was fixed and applied to the Ubuntu package but was not available in RHEL or CENTOS. Not even Debian had the fix in it's package, of the same 'major' version and the equivalent subversion was not available on any distro, with the exception of Ubuntu.

I hope this is not the case as I work in a RedHat shop.

Are you running 1.2.18 or 1.2.20 or a later version?

Thanks!

pitbulk commented 9 years ago

I'm using ubuntu 13.10 with xmlsec1 1.2.18 (openssl) where the python-toolkit is working. dm.xmlsec.binding defines those transformations:

TransformInclC14N, TransformInclC14NWithComments, TransformInclC14N11,
TransformInclC14N11WithComments, TransformExclC14N, TransformExclC14NWithComments,
TransformEnveloped, TransformXPath, TransformXPath2, TransformXPointer, TransformXslt,
TransformRemoveXmlTagsC14N, TransformVisa3DHack, TransformAes128Cbc,
TransformAes192Cbc, TransformAes256Cbc, TransformKWAes128, TransformKWAes192, 
TransformKWAes256, TransformDes3Cbc, TransformKWDes3, TransformDsaSha1,
TransformHmacMd5, TransformHmacRipemd160, TransformHmacSha1, 
TransformHmacSha224,TransformHmacSha256, TransformHmacSha384, 
TransformHmacSha512, TransformMd5, TransformRipemd160, TransformRsaMd5, 
TransformRsaRipemd160, TransformRsaSha1, TransformRsaSha224, TransformRsaSha256, 
TransformRsaSha384, TransformRsaSha512, TransformRsaPkcs1, TransformRsaOaep, 
TransformSha1, TransformSha224, TransformSha256, TransformSha384, TransformSha512

I think we need to compare the /usr/include/xmlsec1/xmlsec/transforms.h from the ubuntu version from the 1.2.18 with the one that is installed using rhel/centOS to see what transform does not exists in the rhel/centOS branch.

This is the ubuntu version: https://gist.github.com/pitbulk/ab258762a9487dc7030e that is the same that the lastest xmlsec version: https://www.aleksey.com/xmlsec/download.html

pitbulk commented 9 years ago

@navneil and @cjsteinke There is a new release of the dm.xmlsec.binding library.

Can you please check if this new release solve your problems?

navneil commented 9 years ago

I gave it a try and it sill doesn't work :(

FxChiP commented 9 years ago

Can confirm the transforms.h given in the gist matches the transforms.h on CentOS 6 save a trailing newline, still experiencing this issue.

FxChiP commented 9 years ago

Can confirm https://bugs.launchpad.net/lxml/+bug/928333 is unrelated in my case

FxChiP commented 9 years ago

All transforms listed are present in the various xmlsec1 header files on CentOS 6 as well.

FxChiP commented 9 years ago

There are two transforms referenced by the binding that do have NULLs in the xmlsec1 library: TransformRemoveXmlTagsC14N and TransformVisa3DHack. However, xmlChar2py will return None on NULL pointers.

FxChiP commented 9 years ago

Simply importing the shared object is enough:

>>> import dm.xmlsec.binding._xmlsec

Program received signal SIGSEGV, Segmentation fault.
__pyx_f_4lxml_5etree_funicode (__pyx_v_s=0x10 <Address 0x10 out of bounds>)
    at src/lxml/lxml.etree.c:26753
26753   src/lxml/lxml.etree.c: No such file or directory.
    in src/lxml/lxml.etree.c
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.132.el6_5.4.x86_64 keyutils-libs-1.4-5.el6.x86_64 krb5-libs-1.10.3-33.el6.x86_64 libcom_err-1.41.12-21.el6.x86_64 libgcrypt-1.4.5-11.el6_4.x86_64 libgpg-error-1.7-4.el6.x86_64 libselinux-2.0.94-5.8.el6.x86_64 libtool-ltdl-2.2.6-15.5.el6.x86_64 libxml2-2.7.6-17.el6_6.1.x86_64 libxslt-1.1.26-2.el6_3.1.x86_64 openssl-1.0.1e-30.el6_6.5.x86_64 xmlsec1-1.2.20-4.el6.x86_64 xmlsec1-openssl-1.2.20-4.el6.x86_64 zlib-1.2.3-29.el6.x86_64
(gdb) bt
#0  __pyx_f_4lxml_5etree_funicode (__pyx_v_s=0x10 <Address 0x10 out of bounds>)
    at src/lxml/lxml.etree.c:26753
#1  0x00007fffef8a55fe in pyunicode (__pyx_v_s=<value optimized out>)
    at src/lxml/lxml.etree.c:179019
#2  0x00007ffff1e0dcc7 in __pyx_f_2dm_6xmlsec_7binding_7_xmlsec_xmlChar2py (
    o=<value optimized out>, x=<value optimized out>) at src/_xmlsec.c:8352
#3  __pyx_pf_2dm_6xmlsec_7binding_7_xmlsec_9Transform_4href___get__ (
    o=<value optimized out>, x=<value optimized out>) at src/_xmlsec.c:2421
#4  __pyx_pw_2dm_6xmlsec_7binding_7_xmlsec_9Transform_4href_1__get__ (
    o=<value optimized out>, x=<value optimized out>) at src/_xmlsec.c:2399
#5  __pyx_getprop_2dm_6xmlsec_7binding_7_xmlsec_9Transform_href (
    o=<value optimized out>, x=<value optimized out>) at src/_xmlsec.c:8651
#6  0x0000000000451d12 in _PyObject_GenericGetAttrWithDict (
    obj=0x7ffff7f70ac8, name=<value optimized out>) at Objects/object.c:1434
#7  PyObject_GenericGetAttr (obj=0x7ffff7f70ac8, name=<value optimized out>)
    at Objects/object.c:1456
#8  0x00000000004a4a37 in PyEval_EvalFrameEx (f=<value optimized out>, 
    throwflag=<value optimized out>) at Python/ceval.c:2256
#9  0x000000000050885d in gen_send_ex (gen=0x7fffee734550, arg=0x0, 
    exc=<value optimized out>) at Objects/genobject.c:84
#10 0x00000000004188cb in PyIter_Next (iter=<value optimized out>)
    at Objects/abstract.c:3107
...

jborg commented 9 years ago

It looks like /usr/bin/xmlsec1-config is broken on CentOS 6 (at least). The following changes fixes this for me:

https://gist.github.com/jborg/d50975951580c53322a0

xmlsec1-config --cflags needs to include "-DXMLSEC_NO_SIZE_T" on 64 bit platforms, but does not.

"pkg-config --cflags xmlsec1-openssl" seems to work though.

pitbulk commented 9 years ago

@jborg Thanks, anyone else can confirm?

AntoineButeau commented 9 years ago

I'm on RedHat and when trying to install dm.xmlsec.binding library i get this error: Error : cannot get XMLSec1 pre-processor and compiler flags; do you have the libxmlsec1 development package installed?

Although I've compiled xmlsec lib, because they are not part of yum anymore. Any idea ?

repagh commented 9 years ago

@AntoineButeau: the xmlsec1-devel package should be in the RHEL optional repository

@pitbulk: I had the same problem, modified my xmlsec1-config to add the "-DXMLSEC_NO_SIZE_T", and got the python-saml flask demo to work on Fedora 20.

AntoineButeau commented 9 years ago

Thanks @repagh did that previously but wasn't working which is strange. I'll try a couple more things and I'll see.

repagh commented 9 years ago

I needed to remove and reinstall. Xmlsec On Mar 12, 2015 6:14 PM, "Antoine Buteau" notifications@github.com wrote:

Thanks @repagh https://github.com/repagh did that previously but wasn't working which is strange. I'll try a couple more things and I'll see.

— Reply to this email directly or view it on GitHub https://github.com/onelogin/python-saml/issues/30#issuecomment-78532806.

serpulga commented 9 years ago

I'm also having problems running dm.xmlsec.binding, specifically loading keys:

dsig_ctx.signKey = xmlsec.Key.loadMemory(key, xmlsec.KeyDataFormatPem, None)
    File "_xmlsec.pyx", line 192, in dm.xmlsec.binding._xmlsec.Key.loadMemory (src/_xmlsec.c:2951)
ValueError: failed to load key from memory\n"}

Looks like I can load keys from memory and from file from the python interpreter, but when I place the code inside my Django application, I get the error above.

jjguy commented 9 years ago

Just a +1 here to confirm same behavior on CentOS 6.6 x64 using dm.xmlsec.binding v1.3.2.

After pip install python-saml, I got:

[root@localhost demo-flask]# python index.py
Segmentation fault
[root@localhost demo-flask]#

Edited /usr/bin/xmlsec1-config per gist linked above, then

[root@localhost bin]# pip uninstall dm.xmlsec.binding
[root@localhost bin]# pip install dm.xmlsec.binding

And now:

[root@localhost demo-flask]# python index.py
 * Running on http://0.0.0.0:8000/ (Press CTRL+C to quit)
 * Restarting with stat

kramimus commented 9 years ago

Also confirming that @jborg's -DXMLSEC_NO_SIZE_T fix works on Amazon Linux.

Any idea if there will be an upstream fix in dm.xmlsec.binding or xmlsec1 at some point?

lawrencexia commented 9 years ago

Trying to get this working on a docker image (centos6), followed @jborg 's advice and modified the xmlsec1-config line 202 to

the_flags="$the_flags  -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_DL_LIBLTDL=1 -DXMLSEC_NO_SIZE_T -I/usr/include/xmlsec1   $the_xml_flags $the_xslt_flags $the_crypto_flags"

after installing the xmlsec1-devel package.

Still getting

Error: cannot get XMLSec1 pre-processor and compiler flags; do you have the `libxmlsec1` development package installed?

while trying to install xm.xmlsec.binding v 1.3.2. Is there something else I am overlooking? Thanks

jjguy commented 9 years ago

@Zoogie looks like you need to yum install xmlsec1-devel.

I was building for a Docker image, too. Here's the relevant steps from my dockerfile, almost verbatim:

FROM centos:6
#Install required packages
RUN yum -y install openssl \
    openssl-devel \
    python \
    python-setuptools \
    python-devel \
    xmlsec1 \
    libffi-devel \
    libxml2-devel \
    xmlsec1-devel \
    xmlsec1-openssl-devel 

# Temp workaround for python sec1 binding used for SAML -- see https://github.com/onelogin/python-saml/issues/30#issuecomment-97531811
RUN sed -i 's/LIBLTDL=1 -I/LIBLTDL=1 -DXMLSEC_NO_SIZE_T -I/' /usr/bin/xmlsec1-config
RUN pip install python-saml

lawrencexia commented 9 years ago

@jjguy awesome, thanks! I had something a long those lines, except I manually ran an ADD to overwrite the xmlsec1-config to have the correct flags, but then file permissions.

RUN chmod 777 /usr/bin/xmlsec1-config

fixed it.

I also had to run

RUN yum -y install libtool-ltdl-devel

to get everything working.

ghost commented 9 years ago

File "_xmlsec.pyx", line 192, in dm.xmlsec.binding._xmlsec.Key.loadMemory (src/_xmlsec.c:2951) ValueError: failed to load key from memory

I got a similar error and found that it was because key was generated using a passphrase. Keys should be generated without a passphrase for this part to work correctly.

jbwyme commented 9 years ago

I was getting the same errors but the following worked for me on Ubuntu 12.04:

sudo apt-get install libxmlsec1-dev swig
pip install python-saml

helmus commented 9 years ago

~~I also got a segmentation fault when importing from onelogin.saml2.auth import OneLogin_Saml2_Auth~~

~~I commented out this line in dm/xmlsec/binding/init.py, per @navneil instructions.~~

~~transformByHref = dict((t.href, t) for t in transforms)~~

~~And now it seems to work.~~ ~~Can someone please clarify the ramifications when commenting out that line ?~~

This was resolved after updating system packages and flipping the DXMLSEC_NO_SIZE_T flag as mentioned above.

vdenotaris commented 8 years ago

I just had the same problem on CentOS7. I tried all the fixes above, but no one seems to work!

rskumar commented 8 years ago

Same issue with me. Please consider migrating to other reliable dependencies. On production, applying these are painful and still no clue how it will work.

pitbulk commented 8 years ago

If you experiencing issues, try https://github.com/onelogin/python3-saml

repagh commented 8 years ago

Just hit this again on a really fresh CentOS7, trying to install python-saml in a virtualenv.

After a failed attempt, it is important to clean python-saml 's dependencies too. I think it might only be dm.xmlsec.binding, but for safety's sake I did a:

pip uninstall python-saml dm.xmlsec.binding isodate defusedxm setuptools lxml

added the -DXMLSEC_NO_SIZE_T to the the_flags variable in /usr/bin/xmlsec1-config

and re installed:

pip install python-saml

testing:

python
Python 2.7.5 (default, Jun 24 2015, 00:41:19) 
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import dm.xmlsec.binding._xmlsec
>>>

asottile commented 8 years ago

This is not usable on lucid (yeah it's old, but we're stuck on it for a while):

Dockerfile to reproduce:

FROM ubuntu:lucid

RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
        python-software-properties && \
    apt-get clean && \
    add-apt-repository ppa:fkrull/deadsnakes && \
    apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
        gcc \
        python2.7-dev \
        libxmlsec1-dev \
        libxml2-dev \
        libxslt1-dev \
        python-pip && \
    apt-get clean

RUN pip install virtualenv pip --upgrade

CMD [ \
    "bash", \
    "-c", \
    "virtualenv venv -ppython2.7 && ./venv/bin/pip install -v python-saml" \
]

Output:

$ docker build -t test . && docker run -ti test
...
  Running setup.py bdist_wheel for dm.xmlsec.binding ...   Destination directory: /tmp/tmpAwQGcxpip-wheel-
  Running command /venv/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-_m3nQK/dm.xmlsec.binding/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpAwQGcxpip-wheel- --python-tag cp27
error
  Complete output from command /venv/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-_m3nQK/dm.xmlsec.binding/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpAwQGcxpip-wheel- --python-tag cp27:
  /usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_requires'
    warnings.warn(msg)
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-2.7
  creating build/lib.linux-x86_64-2.7/dm
  copying dm/__init__.py -> build/lib.linux-x86_64-2.7/dm
  creating build/lib.linux-x86_64-2.7/dm/xmlsec
  copying dm/xmlsec/__init__.py -> build/lib.linux-x86_64-2.7/dm/xmlsec
  creating build/lib.linux-x86_64-2.7/dm/xmlsec/binding
  copying dm/xmlsec/binding/tmpl.py -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding
  copying dm/xmlsec/binding/__init__.py -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding
  copying dm/xmlsec/binding/tests.py -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding
  running egg_info
  writing requirements to dm.xmlsec.binding.egg-info/requires.txt
  writing dm.xmlsec.binding.egg-info/PKG-INFO
  writing namespace_packages to dm.xmlsec.binding.egg-info/namespace_packages.txt
  writing top-level names to dm.xmlsec.binding.egg-info/top_level.txt
  writing dependency_links to dm.xmlsec.binding.egg-info/dependency_links.txt
  warning: manifest_maker: standard file '-c' not found

  reading manifest file 'dm.xmlsec.binding.egg-info/SOURCES.txt'
  writing manifest file 'dm.xmlsec.binding.egg-info/SOURCES.txt'
  copying dm/xmlsec/binding/LICENSE.txt -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding
  copying dm/xmlsec/binding/README.txt -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding
  copying dm/xmlsec/binding/VERSION.txt -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding
  copying dm/xmlsec/binding/tests.txt -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding
  creating build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/deskey.bin -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/encrypt-element-tmpl.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/encrypt1-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/encrypt1-tmpl.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/encrypt2-doc.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/encrypt2-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/encrypt3-doc.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/encrypt3-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/rootcert.pem -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/rsacert.pem -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/rsakey.pem -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/rsapub.pem -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/sign1-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/sign1-tmpl.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/sign2-doc.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/sign2-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/sign3-doc.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/sign3-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/verify4-bad-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/verify4-bad-tmpl.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/verify4-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  copying dm/xmlsec/binding/resources/verify4-tmpl.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
  running build_ext
  building 'dm.xmlsec.binding._xmlsec' extension
  creating build/temp.linux-x86_64-2.7
  creating build/temp.linux-x86_64-2.7/src
  gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_NO_SIZE_T=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_OPENSSL_098=1 -DXMLSEC_CRYPTO_OPENSSL=1 -DXMLSEC_CRYPTO=openssl -I/tmp/pip-build-_m3nQK/dm.xmlsec.binding/.eggs/lxml-3.5.0-py2.7-linux-x86_64.egg/lxml -I/tmp/pip-build-_m3nQK/dm.xmlsec.binding/.eggs/lxml-3.5.0-py2.7-linux-x86_64.egg/lxml/includes -Isrc -I/usr/include/libxml2 -I/usr/include/xmlsec1 -I/usr/include/python2.7 -c src/_xmlsec.c -o build/temp.linux-x86_64-2.7/src/_xmlsec.o
  src/_xmlsec.c: In function '__pyx_pf_2dm_6xmlsec_7binding_7_xmlsec_6EncCtx_6encryptXml':
  src/_xmlsec.c:7267: error: 'XMLSEC_ENC_RETURN_REPLACED_NODE' undeclared (first use in this function)
  src/_xmlsec.c:7267: error: (Each undeclared identifier is reported only once
  src/_xmlsec.c:7267: error: for each function it appears in.)
  src/_xmlsec.c:7312: error: 'struct _xmlSecEncCtx' has no member named 'replacedNodeList'
  src/_xmlsec.c:7348: error: 'struct _xmlSecEncCtx' has no member named 'replacedNodeList'
  src/_xmlsec.c: In function '__pyx_pf_2dm_6xmlsec_7binding_7_xmlsec_6EncCtx_10decrypt':
  src/_xmlsec.c:7830: error: 'XMLSEC_ENC_RETURN_REPLACED_NODE' undeclared (first use in this function)
  src/_xmlsec.c:7875: error: 'struct _xmlSecEncCtx' has no member named 'replacedNodeList'
  src/_xmlsec.c:7911: error: 'struct _xmlSecEncCtx' has no member named 'replacedNodeList'
  src/_xmlsec.c: In function 'init_xmlsec':
  src/_xmlsec.c:10917: error: 'xmlSecTransformInclC14N11Id' undeclared (first use in this function)
  src/_xmlsec.c:10929: error: 'xmlSecTransformInclC14N11WithCommentsId' undeclared (first use in this function)
  error: command 'gcc' failed with exit status 1

  ----------------------------------------
  Failed building wheel for dm.xmlsec.binding
  Running setup.py clean for dm.xmlsec.binding
  Running command /venv/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-_m3nQK/dm.xmlsec.binding/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" clean --all
  Running setup.py bdist_wheel for lxml ...   Destination directory: /tmp/tmpms31CLpip-wheel-
  Running command /venv/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-_m3nQK/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpms31CLpip-wheel- --python-tag cp27
done
  Stored in directory: /.cache/pip/wheels/2d/f6/d6/158b79cc088e551305968e312953c9767855a36f39d4e16054
  Removing source in /tmp/pip-build-_m3nQK/lxml
Successfully built isodate defusedxml lxml
Failed to build dm.xmlsec.binding
Installing collected packages: isodate, defusedxml, lxml, dm.xmlsec.binding, python-saml

  Running setup.py install for dm.xmlsec.binding ...     Running command /venv/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-_m3nQK/dm.xmlsec.binding/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-NB5UvC-record/install-record.txt --single-version-externally-managed --compile --install-headers /venv/include/site/python2.7/dm.xmlsec.binding
error
    Complete output from command /venv/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-_m3nQK/dm.xmlsec.binding/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-NB5UvC-record/install-record.txt --single-version-externally-managed --compile --install-headers /venv/include/site/python2.7/dm.xmlsec.binding:
    /usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_requires'
      warnings.warn(msg)
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/dm
    copying dm/__init__.py -> build/lib.linux-x86_64-2.7/dm
    creating build/lib.linux-x86_64-2.7/dm/xmlsec
    copying dm/xmlsec/__init__.py -> build/lib.linux-x86_64-2.7/dm/xmlsec
    creating build/lib.linux-x86_64-2.7/dm/xmlsec/binding
    copying dm/xmlsec/binding/tmpl.py -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding
    copying dm/xmlsec/binding/__init__.py -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding
    copying dm/xmlsec/binding/tests.py -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding
    running egg_info
    writing requirements to dm.xmlsec.binding.egg-info/requires.txt
    writing dm.xmlsec.binding.egg-info/PKG-INFO
    writing namespace_packages to dm.xmlsec.binding.egg-info/namespace_packages.txt
    writing top-level names to dm.xmlsec.binding.egg-info/top_level.txt
    writing dependency_links to dm.xmlsec.binding.egg-info/dependency_links.txt
    warning: manifest_maker: standard file '-c' not found

    reading manifest file 'dm.xmlsec.binding.egg-info/SOURCES.txt'
    writing manifest file 'dm.xmlsec.binding.egg-info/SOURCES.txt'
    copying dm/xmlsec/binding/LICENSE.txt -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding
    copying dm/xmlsec/binding/README.txt -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding
    copying dm/xmlsec/binding/VERSION.txt -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding
    copying dm/xmlsec/binding/tests.txt -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding
    creating build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/deskey.bin -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/encrypt-element-tmpl.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/encrypt1-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/encrypt1-tmpl.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/encrypt2-doc.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/encrypt2-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/encrypt3-doc.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/encrypt3-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/rootcert.pem -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/rsacert.pem -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/rsakey.pem -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/rsapub.pem -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/sign1-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/sign1-tmpl.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/sign2-doc.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/sign2-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/sign3-doc.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/sign3-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/verify4-bad-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/verify4-bad-tmpl.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/verify4-res.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    copying dm/xmlsec/binding/resources/verify4-tmpl.xml -> build/lib.linux-x86_64-2.7/dm/xmlsec/binding/resources
    running build_ext
    building 'dm.xmlsec.binding._xmlsec' extension
    creating build/temp.linux-x86_64-2.7
    creating build/temp.linux-x86_64-2.7/src
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -D__XMLSEC_FUNCTION__=__FUNCTION__ -DXMLSEC_NO_SIZE_T=1 -DXMLSEC_NO_XKMS=1 -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_OPENSSL_098=1 -DXMLSEC_CRYPTO_OPENSSL=1 -DXMLSEC_CRYPTO=openssl -I/venv/local/lib/python2.7/site-packages/lxml -I/venv/local/lib/python2.7/site-packages/lxml/includes -Isrc -I/usr/include/libxml2 -I/usr/include/xmlsec1 -I/usr/include/python2.7 -c src/_xmlsec.c -o build/temp.linux-x86_64-2.7/src/_xmlsec.o
    src/_xmlsec.c: In function '__pyx_pf_2dm_6xmlsec_7binding_7_xmlsec_6EncCtx_6encryptXml':
    src/_xmlsec.c:7267: error: 'XMLSEC_ENC_RETURN_REPLACED_NODE' undeclared (first use in this function)
    src/_xmlsec.c:7267: error: (Each undeclared identifier is reported only once
    src/_xmlsec.c:7267: error: for each function it appears in.)
    src/_xmlsec.c:7312: error: 'struct _xmlSecEncCtx' has no member named 'replacedNodeList'
    src/_xmlsec.c:7348: error: 'struct _xmlSecEncCtx' has no member named 'replacedNodeList'
    src/_xmlsec.c: In function '__pyx_pf_2dm_6xmlsec_7binding_7_xmlsec_6EncCtx_10decrypt':
    src/_xmlsec.c:7830: error: 'XMLSEC_ENC_RETURN_REPLACED_NODE' undeclared (first use in this function)
    src/_xmlsec.c:7875: error: 'struct _xmlSecEncCtx' has no member named 'replacedNodeList'
    src/_xmlsec.c:7911: error: 'struct _xmlSecEncCtx' has no member named 'replacedNodeList'
    src/_xmlsec.c: In function 'init_xmlsec':
    src/_xmlsec.c:10917: error: 'xmlSecTransformInclC14N11Id' undeclared (first use in this function)
    src/_xmlsec.c:10929: error: 'xmlSecTransformInclC14N11WithCommentsId' undeclared (first use in this function)
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Cleaning up...
  Removing source in /tmp/pip-build-_m3nQK/dm.xmlsec.binding
Command "/venv/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-_m3nQK/dm.xmlsec.binding/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-NB5UvC-record/install-record.txt --single-version-externally-managed --compile --install-headers /venv/include/site/python2.7/dm.xmlsec.binding" failed with error code 1 in /tmp/pip-build-_m3nQK/dm.xmlsec.binding
Exception information:
Traceback (most recent call last):
  File "/venv/local/lib/python2.7/site-packages/pip/basecommand.py", line 209, in main
    status = self.run(options, args)
  File "/venv/local/lib/python2.7/site-packages/pip/commands/install.py", line 317, in run
    prefix=options.prefix_path,
  File "/venv/local/lib/python2.7/site-packages/pip/req/req_set.py", line 731, in install
    **kwargs
  File "/venv/local/lib/python2.7/site-packages/pip/req/req_install.py", line 890, in install
    spinner=spinner,
  File "/venv/local/lib/python2.7/site-packages/pip/utils/__init__.py", line 687, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command "/venv/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-_m3nQK/dm.xmlsec.binding/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-NB5UvC-record/install-record.txt --single-version-externally-managed --compile --install-headers /venv/include/site/python2.7/dm.xmlsec.binding" failed with error code 1 in /tmp/pip-build-_m3nQK/dm.xmlsec.binding

pitbulk commented 8 years ago

@asottile

Not related with Lucid, now I see problems on Ubuntu 14.04.3 and installing specific libraries.

I see same bug with xmlsec-0.3.0 and Cython-0.23.4

Installed /tmp/easy_install-Wm1qCy/xmlsec-0.3.0/.eggs/Cython-0.23.4-py2.7-linux-x86_64.egg
src/xmlsec/enc.c: In function ‘__pyx_import_star_set’:
src/xmlsec/enc.c:3384:37: error: lvalue required as left operand of assignment
     XMLSEC_ENC_RETURN_REPLACED_NODE = __Pyx_PyInt_As_unsigned_int(o); if ((XMLSEC_ENC_RETURN_REPLACED_NODE == (unsigned int)-1) && PyErr_Occurred()) {__pyx_filename = __pyx_f[2]; __pyx_lineno = 8; __pyx_clineno = __LINE__; goto __pyx_L2_error;};
                                     ^
error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Try

pip install lxml==3.5.0b1
pip install Cython==0.23

and then execute the pip install python-saml

sunmoonone commented 8 years ago

confirmming @jborg's -DXMLSEC_NO_SIZE_T works on centos 7 do not use pip cache is import while reinstalling dm.xmlsec.binding:

pip uninstall dm.xmlsec.binding
pip install --no-cache-dir dm.xmlsec.binding

tsani commented 8 years ago

Just weighing in that @jborg's gist involving -DXMLSEC_NO_SIZE_T works on centos 6.2 for me as well!

SAML-Toolkits / python-saml

dm.xmlsec.binding broken? #30