rpm build with python-2.7 on rhel6/toss2

[garlick]

There is an LLNL provided package for python2.7 on our systems that installs into /opt with a module config file named "python/2.7". However it doesn't look like we've packaged libffi or cffi.

I think we'll need to provide these as RPM's in order to build a flux RPM with python enabled.

[garlick]

I misspoke, libffi is part of RHEL6 and installed on our systems (hype at least):

libffi-devel-3.0.5-3.2.el6.x86_64
libffi-3.0.5-3.2.el6.x86_64

So maybe it's just a question of providing a python-cffi package v 1.1 or later?

[garlick]

For reference, packages installed on our rhel7/toss3 test systems (upgrade2.llnl.gov) are:

python-2.7.5-16.el7.x86_64
libffi-3.0.13-11.el7.x86_64
libffi-devel-3.0.13-11.el7.x86_64

python-cffi is not installed; 0.8.6 is available from EPEL but we need 1.1+.

[trws]

Thankfully, cffi is a well built setuptools package, so building a basic RPM just takes:

$ python setup.py bdist_rpm

That will build source, binary, and debug RPMs along with a tarball into ./dist. I tried it in my environment just to be sure it would work, so I have some built if you want them. Mine may or may not be toss-compatible though, since I use a non-redhat python to avoid dealing with the mismatched unicode width between the interpreter and libpython in our default python.

[grondo]

I wonder if that would have proper buildrequires for a mock build. Sometimes you can just grab the upstream fedora pacakge

https://kojipkgs.fedoraproject.org//packages/python-cffi/1.1.2/3.fc24/src/python-cffi-1.1.2-3.fc24.src.rpm

(Sometimes these can't be built on older RHEL versions though)

However, since this version isn't even in RHEL7 yet I wonder if this requirement will be a pain point for users. Is there a reason 0.8.6 couldn't be supported? (Not a big deal, but I thought it could not hurt to ask)

[trws]

We could drop it to 1.0 relatively easily if that would help, but below that would be challenging. That is the version that gained the ability to construct an external C file at build time to generate the module. Before that a python script had to be run that would generate and compile the module inline.

Sent with Good (www.good.com)

---

From: Mark Grondona Sent: Wednesday, July 22, 2015 12:56:18 PM To: flux-framework/flux-core Cc: Scogland, Thomas Richard William Subject: Re: [flux-core] rpm build with python-2.7 on rhel6/toss2 (#281)

I wonder if that would have proper buildrequires for a mock build. Sometimes you can just grab the upstream fedora pacakge

https://kojipkgs.fedoraproject.org//packages/python-cffi/1.1.2/3.fc24/src/python-cffi-1.1.2-3.fc24.src.rpm

(Sometimes these can't be built on older RHEL versions though)

However, since this version isn't even in RHEL7 yet I wonder if this requirement will be a pain point for users. Is there a reason 0.8.6 couldn't be supported? (Not a big deal, but I thought it could not hurt to ask)

— Reply to this email directly or view it on GitHubhttps://github.com/flux-framework/flux-core/issues/281#issuecomment-123845211.

[trws]

@grondo: It does not have BuildRequires set up apparently, but it does output the spec file to allow a custom rpm build. Adding these should be sufficient:

BuildRequires:  libffi-devel python-sphinx
BuildRequires:  python2-devel python-setuptools python-pycparser

sphinx is probably not really required, but allows the docs to build.

[garlick]

I take it python-cffi and its dependencies would need to be built in /opt, against the python-2.7 installed there on rhel6 / toss2? Is it correct to assume that building dependencies against the system python (2.6) would render them unusable with 2.7?

[trws]

I would say that's probable. The interface is sufficiently similar that it probably wouldn't break obviously, but I think some of the default sizes and definitions changed between the two, or at least between the builds on our systems.

Really, I didn't even think about targeting 2.6 with this, because it has been deprecated to the point of receiving no further bug or security patches since 2013. Unlike the cffi issue, it's entirely possible to re-work things to support python 2.6, but it would be a non-trivial amount of work.

[grondo]

@garlick, I assume we should therefore generate flux RPMs targeted at /opt as well? Perhaps for TOSS3/RHEL7 we can move to /usr based packages.

Let me know if you want any help with building the dependencies packages.

[garlick]

Yes, it sounds like we will need those packages in /opt for rhel6, and sooner would be better that later. Any help would be appreciated if you have cycles for this @grondo.

[grondo]

I got back to this issue yesterday and, at least for TOSS 2, supporting python in the flux RPMs is going to be a bit of a pain. I was building a python-cffi package in /opt using the python-2.7 package there, but it turns out python-cffi also needs Cython and pycparser so I'll have to also build those dependencies first. These are also just the python dependencies I know about so far.

I will try for a little while longer building up the dependencies, but I'd like to perhaps explore other alternatives:

1. Build flux RPMs without python for RHEL6/TOSS2 environment. I kind of get the feeling @trws is depending on python bindings in order to build parts of our near-term scheduling tool, so I suspect this is a non-starter for him.
2. Use spack packages for RHEL6/TOSS2 systems. This really bothers me because I don't like the waste and maintainability issues of having every user of flux having their own personal install of the code and all its dependencies. Also how do you handle things like systemwide config files (if we have such a thing in this version of the tool).

I don't have any other good alternatives, so I guess I'll continue sorting out RPM issues. Open to other ideas though...

[trws]

Why do you say it requires cython? The only dependencies are pycparser and pytest, and pytest is only necessary for running the test harness.
I noticed that the RedHat RPM package requires it, but cffi doesn't use it, so I have no idea why that is.

See for example this OpenSUSE RPM's deps (http://rpmfind.net//linux/RPM/opensuse/factory/x86_64/python-cffi-1.1.0-1.2.x86_64.html)

If an RPM for cffi is sufficiently onerous, it would be relatively easy to depend on setuptools/pip, then use pip to perform a "local" install of cffi and its requisite dependencies into our install directory. The downside to this is that it would require an internet connection. A similar alternative would be to package pycparser and cffi with flux, but that has other downsides. They are BSD and MIT licensed respectively, so if that's easiest it wouldn't be too bad, just add a couple of extra targets to run their setup.py install targets alongside ours.

On 5 Aug 2015, at 9:25, Mark Grondona wrote:

I got back to this issue yesterday and, at least for TOSS 2, supporting python in the flux RPMs is going to be a bit of a pain. I was building a python-cffi package in /opt using the python-2.7 package there, but it turns out python-cffi also needs Cython and pycparser so I'll have to also build those dependencies first. These are also just the python dependencies I know about so far.

I will try for a little while longer building up the dependencies, but I'd like to perhaps explore other alternatives:

1. Build flux RPMs without python for RHEL6/TOSS2 environment. I kind of get the feeling @trws is depending on python bindings in order to build parts of our near-term scheduling tool, so I suspect this is a non-starter for him.
2. Use spack packages for RHEL6/TOSS2 systems. This really bothers me because I don't like the waste and maintainability issues of having every user of flux having their own personal install of the code and all its dependencies. Also how do you handle things like systemwide config files (if we have such a thing in this version of the tool).

I don't have any other good alternatives, so I guess I'll continue sorting out RPM issues. Open to other ideas though...

---

Reply to this email directly or view it on GitHub: https://github.com/flux-framework/flux-core/issues/281#issuecomment-128058251

[grondo]

Ah, my mistake was trusting the upstream RPM for python-cffi ;-)

But I will need to make a python-2.7 version of pycparser, or will the version packaged with RHEL6 work?

[trws]

The one from RHEL6 should just work, I don't think there have been any major feature changes there for quite a while. If you do need to make a package though, it's a pretty simple one since it doesn't have any dependencies outside the standard libraries.

On 5 Aug 2015, at 9:53, Mark Grondona wrote:

Ah, my mistake was trusting the upstream RPM for python-cffi ;-)

But I will need to make a python-2.7 version of pycparser, or will the version packaged with RHEL6 work?

---

Reply to this email directly or view it on GitHub: https://github.com/flux-framework/flux-core/issues/281#issuecomment-128068898

[grondo]

Hm, I'm feeling pretty lame. Inside the mock builder, I can't even get setuptools to work. Currently the RHEL6 version of setuptools is pulled in, but we're using the python from /opt/python-2.7.

Any hints?

DEBUG: + module load python/2.7
DEBUG: ++ /usr/bin/modulecmd sh load python/2.7
DEBUG: + eval LD_LIBRARY_PATH=/opt/python-2.7/lib ';export' 'LD_LIBRARY_PATH;LOADEDMODULES=python/2.7' ';export' 'LOADEDMODULES;MANPATH=/opt/python-2.7/share/man:' ';export' 'MANPATH;PATH=/opt/python-2.7/bin:/usr/lib64/ccache:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin:/builddir/bin' ';export' 'PATH;_LMFILES_=/opt/modules/modulefiles/python/2.7' ';export' '_LMFILES_;'
DEBUG: ++ LD_LIBRARY_PATH=/opt/python-2.7/lib
DEBUG: ++ export LD_LIBRARY_PATH
DEBUG: ++ LOADEDMODULES=python/2.7
DEBUG: ++ export LOADEDMODULES
DEBUG: ++ MANPATH=/opt/python-2.7/share/man:
DEBUG: ++ export MANPATH
DEBUG: ++ PATH=/opt/python-2.7/bin:/usr/lib64/ccache:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin:/builddir/bin
DEBUG: ++ export PATH
DEBUG: ++ _LMFILES_=/opt/modules/modulefiles/python/2.7
DEBUG: ++ export _LMFILES_
DEBUG: + export PYTHONPATH=/opt/python-2.7/lib/python2.7
DEBUG: + PYTHONPATH=/opt/python-2.7/lib/python2.7
DEBUG: + python setup.py build --prefix=/opt/python2.7-cffi-1.1.2
DEBUG: Traceback (most recent call last):
DEBUG:   File "setup.py", line 118, in <module>
DEBUG:     from setuptools import setup, Distribution, Extension
DEBUG: ImportError: No module named setuptools
DEBUG: RPM build errors:
DEBUG: error: Bad exit status from /var/tmp/rpm-tmp.tIst4P (%build)
DEBUG:     Bad exit status from /var/tmp/rpm-tmp.tIst4P (%build)
DEBUG: Child return code was: 1

[trws]

Ah... Right, the RPM will have put setup tools into the site-packages directory for the system python rather than the opt python. It may be sufficient to install the same setuptools RPM into the target path in opt.

On 5 Aug 2015, at 10:29, Mark Grondona wrote:

Hm, I'm feeling pretty lame. Inside the mock builder, I can't even get setuptools to work. Currently the RHEL6 version of setuptools is pulled in, but we're using the python from /opt/python-2.7.

Any hints?

DEBUG: + module load python/2.7
DEBUG: ++ /usr/bin/modulecmd sh load python/2.7
DEBUG: + eval LD_LIBRARY_PATH=/opt/python-2.7/lib ';export' 
'LD_LIBRARY_PATH;LOADEDMODULES=python/2.7' ';export' 
'LOADEDMODULES;MANPATH=/opt/python-2.7/share/man:' ';export' 
'MANPATH;PATH=/opt/python-2.7/bin:/usr/lib64/ccache:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin:/builddir/bin' 
';export' 'PATH;_LMFILES_=/opt/modules/modulefiles/python/2.7' 
';export' '_LMFILES_;'
DEBUG: ++ LD_LIBRARY_PATH=/opt/python-2.7/lib
DEBUG: ++ export LD_LIBRARY_PATH
DEBUG: ++ LOADEDMODULES=python/2.7
DEBUG: ++ export LOADEDMODULES
DEBUG: ++ MANPATH=/opt/python-2.7/share/man:
DEBUG: ++ export MANPATH
DEBUG: ++ 
PATH=/opt/python-2.7/bin:/usr/lib64/ccache:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin:/builddir/bin
DEBUG: ++ export PATH
DEBUG: ++ _LMFILES_=/opt/modules/modulefiles/python/2.7
DEBUG: ++ export _LMFILES_
DEBUG: + export PYTHONPATH=/opt/python-2.7/lib/python2.7
DEBUG: + PYTHONPATH=/opt/python-2.7/lib/python2.7
DEBUG: + python setup.py build --prefix=/opt/python2.7-cffi-1.1.2
DEBUG: Traceback (most recent call last):
DEBUG:   File "setup.py", line 118, in <module>
DEBUG:     from setuptools import setup, Distribution, Extension
DEBUG: ImportError: No module named setuptools
DEBUG: RPM build errors:
DEBUG: error: Bad exit status from /var/tmp/rpm-tmp.tIst4P (%build)
DEBUG:     Bad exit status from /var/tmp/rpm-tmp.tIst4P (%build)
DEBUG: Child return code was: 1

---

Reply to this email directly or view it on GitHub: https://github.com/flux-framework/flux-core/issues/281#issuecomment-128084236

[grondo]

Ok, I made some progress and I have python2.7-setuptools and python2.7-cffi built for TOSS.

However, it appears that our RPM of python-2.7 is built without devel support, because there is no libpython.so, and the flux build bails here:

make[3]: Entering directory `/home/grondo/flux-core.git/src/modules/pymod'
  CC     py_mod.lo
  CCLD   pymod.la
/usr/bin/ld: /opt/python-2.7/lib/libpython2.7.a(abstract.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC
/opt/python-2.7/lib/libpython2.7.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
make[3]: *** [pymod.la] Error 1
make[3]: Leaving directory `/home/grondo/flux-core.git/src/modules/pymod'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/grondo/flux-core.git/src/modules'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/grondo/flux-core.git/src'
make: *** [all-recursive] Error 1

The /opt version of the python-2.7 rpm basically does

./configure --prefix=/opt/%{name}
make install

I suppose there is some missing option to configure to build and/or install the shared libs? In any event I'll have to rebuild the python-2.7 rpm to make any more progres...

[grondo]

Ok, I got the above sorted out, now the build fails here:

/opt/python-2.7/bin/python ../../../../src/bindings/python/make_binding.py --path ../../../../src/common/libflux \
                  --package flux \
                              --modname _core \
                  --add_sub '.*va_list.*|||' \
                  --ignore_header 'handle_impl' \
                  --add_long_sub 'FLUX_SEC_TYPE_ALL.*\n.*\),|||FLUX_SEC_TYPE_ALL = 7,'\
                  flux.h
/opt/python-2.7/bin/python _core_build.py
Traceback (most recent call last):
  File "_core_build.py", line 3, in <module>
    ffi = FFI()
  File "/opt/python2.7-cffi-1.1.2/lib/python2.7/site-packages/cffi/api.py", line 52, in __init__
    from . import cparser, model
  File "/opt/python2.7-cffi-1.1.2/lib/python2.7/site-packages/cffi/cparser.py", line 6, in <module>
    import pycparser
ImportError: No module named pycparser

Do you know the package including "pycparser"?

[trws]

It doesn't seem to be packaged by default for el6, but here's a centos 7 package for it centos.org.

[grondo]

There are EPEL packages for python-pycparser and its dependency python-ply. Hopefully these don't also have to be built as /opt packages.

[grondo]

Ok, I got this working just before leaving today -- with one caveat: the version of python-2.7 built for /opt will need an update to include libpython2.7.so (for some reason the python build didn't include this by default)

Then we'll need to include the new packages python2.7-{ply,pycparser,cffi} on our systems in addition to python-2.7, and before a build just do

module load python-2.7 python2.7-ply python2.7-pycparser python2.7-cffi

and python bindings are successfully built.

In order to fully satisfy this issue, however, I'll still need to build flux rpms for /opt...

[trws]

Thanks for going through all this Mark. I hadn't intended to make it such a hefty dependency. =/

Sent with Good (www.good.com)

---

From: Mark Grondona Sent: Friday, August 07, 2015 5:07:15 PM To: flux-framework/flux-core Cc: Scogland, Thomas Richard William Subject: Re: [flux-core] rpm build with python-2.7 on rhel6/toss2 (#281)

Ok, I got this working just before leaving today -- with one caveat: the version of python-2.7 built for /opt will need an update to include libpython2.7.so (for some reason the python build didn't include this by default)

Then we'll need to include the new packages python2.7-{ply,pycparser,cffi} on our systems in addition to python-2.7, and before a build just do

module load python-2.7 python2.7-ply python2.7-pycparser python2.7-cffi

and python bindings are successfully built.

In order to fully satisfy this issue, however, I'll still need to build flux rpms for /opt...

— Reply to this email directly or view it on GitHubhttps://github.com/flux-framework/flux-core/issues/281#issuecomment-128864205.