Including librabbitmq-c necessary?

[fladi]

No really an issue, but is there any rationale on why librabbitmq-c is included in the source? Why not make the installation of the c-library a dependency? The current situation requires quite a lot of changes to your build process if someone (me) wants to shape this into a Debian package.

[ask]

Sadly, there's no way to specify a C library dependency in Python.

The most important thing is keeping the python library easy to install, as it won't see big adoption if there are complex build instructions. rabbitmq-c is quite uncommon in package managers.

[ask]

Btw, don't think you need many changes to depend on a system install of rabbitmq-c, that's just skipping the part where setup.py calls configure and link dynamically instead.

[ask]

Would it be possible to use an environment variable/argument to setup.py? Would that make sense for distro sources?

[fladi]

Sure, that would make it way easier.

[agriffis]

@fladi @ask Did you ever look further into this? I was looking at #71 which also affects me, and I'm wondering if it all gets better by depending on the Fedora-provided librabbitmq-devel (version 0.7.1 presently). I've started looking at setup.py but getting lost in create_builder() so far.

[agriffis]

Looks like @fladi patched the Debian package to build with the distro lib: http://packages.ubuntu.com/vivid/python-librabbitmq and especially debian/patches/fix_setup.patch in http://archive.ubuntu.com/ubuntu/pool/universe/p/python-librabbitmq/python-librabbitmq_1.5.2-1.debian.tar.xz which I'll include here for reading.

Description: Simplify setup.py
 Upstream decided to include a copy of librabbitmq-c and builds it with a
 rather complicated setup.py. This patch removes all the cruft and assumes
 that librabbitmq-c is already present on the system (through the
 librabbitmq-dev package).
 The C extension to this package is built as an ordinary Extension() as used
 by setup().
Author: Michael Fladischer <FladischerMichael@fladi.at>
Last-Update: 2013-06-07
Forwarded: no

--- a/setup.cfg
+++ b/setup.cfg
@@ -1,8 +1,5 @@
 [nosetests]
 where = librabbitmq/tests
-cover3-branch = 1
-cover3-html = 1
-cover3-package = librabbitmq

 [build_sphinx]
 source-dir = docs/
--- a/setup.py
+++ b/setup.py
@@ -1,214 +1,13 @@
-import os
-import platform
-import sys
-from glob import glob
-from setuptools import setup, find_packages
-
-# --with-librabbitmq=<dir>: path to librabbitmq package if needed
-
-LRMQDIST = lambda *x: os.path.join('clib', *x)
-LRMQSRC = lambda *x: LRMQDIST('librabbitmq', *x)
-SPECPATH = lambda *x: os.path.join('rabbitmq-codegen', *x)
-PYCP = lambda *x: os.path.join('Modules', '_librabbitmq', *x)
-
-
-
-def senv(*k__v, **kwargs):
-    sep = kwargs.get('sep', ' ')
-    restore = {}
-    for k, v in k__v:
-        prev = restore[k] = os.environ.get(k)
-        os.environ[k] = (prev + sep if prev else '') + str(v)
-    return dict((k, v) for k, v in restore.iteritems() if v is not None)
-
-
-def codegen():
-    codegen = LRMQSRC('codegen.py')
-    spec = SPECPATH('amqp-rabbitmq-0.9.1.json')
-    sys.path.insert(0, SPECPATH())
-    commands = [
-        (sys.executable, codegen, 'header', spec, LRMQSRC('amqp_framing.h')),
-        (sys.executable, codegen, 'body', spec, LRMQSRC('amqp_framing.c')),
-    ]
-    restore = senv(('PYTHONPATH', SPECPATH()), sep=':')
-    try:
-        for command in commands:
-            print('- generating %r' % command[-1])
-            print(' '.join(command))
-            os.system(' '.join(command))
-    finally:
-        os.environ.update(restore)
-
-
-
-def create_builder():
-    from setuptools import Extension
-    from distutils.command.build import build as _build
-    cmd = None
-    pkgdirs = []  # incdirs and libdirs get these
-    libs = []
-    defs = []
-    incdirs = []
-    libdirs = []
-
-    def append_env(L, e):
-        v = os.environ.get(e)
-        if v and os.path.exists(v):
-            L.append(v)
-
-    append_env(pkgdirs, 'LIBRABBITMQ')
-
-    # Hack up sys.argv, yay
-    unprocessed = []
-    for arg in sys.argv[1:]:
-        if arg == '--gen-setup':
-            cmd = arg[2:]
-        elif '=' in arg:
-            if arg.startswith('--with-librabbitmq='):
-                pkgdirs.append(arg.split('=', 1)[1])
-                continue
-        unprocessed.append(arg)
-    sys.argv[1:] = unprocessed
-
-    incdirs.append(LRMQSRC())
-    PyC_files = map(PYCP, [
-        'connection.c',
-    ])
-    librabbit_files = map(LRMQSRC, [
-        'amqp_api.c',
-        'amqp_connection.c',
-        'amqp_consumer.c',
-        'amqp_framing.c',
-        'amqp_hostcheck.c',
-        'amqp_mem.c',
-        'amqp_socket.c',
-        'amqp_table.c',
-        'amqp_tcp_socket.c',
-        'amqp_timer.c',
-        'amqp_url.c',
-    ])
-
-    incdirs.append(LRMQDIST())  # for config.h
-
-    if is_linux:  # Issue #42
-        libs.append('rt')  # -lrt for clock_gettime
-
-    librabbitmq_ext = Extension('_librabbitmq',
-                            sources=PyC_files + librabbit_files,
-                            libraries=libs, include_dirs=incdirs,
-                            library_dirs=libdirs, define_macros=defs)
-                            #depends=(glob(PYCP('*.h')) + ['setup.py']))
-
-    # Hidden secret: if environment variable GEN_SETUP is set, generate Setup file.
-    if cmd == 'gen-setup':
-        line = ' '.join((
-            librabbitmq_ext.name,
-            ' '.join('-l' + lib for lib in librabbitmq_ext.libraries),
-            ' '.join('-I' + incdir for incdir in librabbitmq_ext.include_dirs),
-            ' '.join('-L' + libdir for libdir in librabbitmq_ext.library_dirs),
-            ' '.join('-D' + name + ('=' + str(value), '')[value is None] for
-                    (name, value) in librabbitmq_ext.define_macros)))
-        open('Setup', 'w').write(line + '\n')
-        sys.exit(0)
-
-    class build(_build):
-        stdcflags = [
-            '-DHAVE_CONFIG_H',
-        ]
-        if os.environ.get('PEDANTIC'):
-            # Python.h breaks -pedantic, so can only use it while developing.
-            stdcflags.append('-pedantic -Werror')
-
-        def run(self):
-            here = os.path.abspath(os.getcwd())
-            H = lambda *x: os.path.join(here, *x)
-            from distutils import sysconfig
-            config = sysconfig.get_config_vars()
-            try:
-                vars = {'ld': config['LDFLAGS'],
-                        'c': config['CFLAGS']}
-                for key in list(vars):
-                    vars[key] = vars[key].replace('-lSystem', '')
-                    # Python on Maverics sets this, but not supported on clang
-                    vars[key] = vars[key].replace('-mno-fused-madd', '')
-                    vars[key] = vars[key].replace(
-                        '-isysroot /Developer/SDKs/MacOSX10.6.sdk', '')
-                    vars[key] = vars[key].replace('-Wall', '')
-                restore = senv(
-                    ('CFLAGS', vars['c']),
-                    ('LDFLAGS', vars['ld']),
-                )
-                try:
-                    os.chdir(LRMQDIST())
-                    if not os.path.isfile('config.h'):
-                        print('- configure rabbitmq-c...')
-                        if os.system('/bin/sh configure --disable-tools \
-                                --disable-docs --disable-dependency-tracking'):
-                            return
-                    #print('- make rabbitmq-c...')
-                    #os.chdir(LRMQSRC())
-                    #os.system(''%s' all' % find_make())
-                finally:
-                    os.environ.update(restore)
-            finally:
-                os.chdir(here)
-            restore = senv(
-                #('LDFLAGS', ' '.join(glob(LRMQSRC('*.o')))),
-                ('CFLAGS', ' '.join(self.stdcflags)),
-            )
-            codegen()
-            try:
-                _build.run(self)
-            finally:
-                os.environ.update(restore)
-    return librabbitmq_ext, build
-
-
-def find_make(alt=('gmake', 'gnumake', 'make', 'nmake')):
-    for path in os.environ['PATH'].split(':'):
-        for make in (os.path.join(path, m) for m in alt):
-            if os.path.isfile(make):
-                return make
-
+from setuptools import setup, find_packages, Extension

 long_description = open('README.rst', 'U').read()
-distmeta = open(PYCP('distmeta.h')).read().strip().splitlines()
+distmeta = open('Modules/_librabbitmq/distmeta.h').read().strip().splitlines()
 distmeta = [item.split('\"')[1] for item in distmeta]
 version = distmeta[0].strip()
 author = distmeta[1].strip()
 contact = distmeta[2].strip()
 homepage = distmeta[3].strip()

-
-
-ext_modules = []
-cmdclass = {}
-packages = []
-goahead = False
-is_jython = sys.platform.startswith('java')
-is_pypy = hasattr(sys, 'pypy_version_info')
-is_py3k = sys.version_info[0] == 3
-is_win = platform.system() == 'Windows'
-is_linux = platform.system() == 'Linux'
-if is_jython or is_pypy or is_py3k or is_win:
-    pass
-elif find_make():
-    try:
-        librabbitmq_ext, build = create_builder()
-    except Exception, exc:
-        print('Could not create builder: %r' % (exc, ))
-        raise
-    else:
-        goahead = True
-        ext_modules= [librabbitmq_ext]
-        cmdclass = {'build': build}
-        packages = find_packages(exclude=['ez_setup', 'tests', 'tests.*'])
-
-if not goahead:
-    ext_modules = []
-    cmdclass = {}
-    packages = []
-
 setup(
     name='librabbitmq',
     version=version,
@@ -220,12 +19,14 @@
     long_description=long_description,
     test_suite='nose.collector',
     zip_safe=False,
-    packages=packages,
-    cmdclass=cmdclass,
-    install_requires=[
-        'amqp>=1.2.1',
+    packages=find_packages(exclude=['ez_setup', 'funtests', 'funtests.*']),
+    package_dir={'librabbitmq.funtests': 'funtests'},
+    ext_modules=[
+        Extension('_librabbitmq', ['Modules/_librabbitmq/connection.c'],
+                  libraries=['rabbitmq'],
+                  include_dirs=['/usr/include'],
+                  library_dirs=['/usr/lib'])
     ],
-    ext_modules=ext_modules,
     classifiers=[
         'Development Status :: 5 - Production/Stable',
         'Operating System :: POSIX',

[jayvdb]

@agriffis, did Debian get tests to pass with that?

I tried the same thing with the recent source, and linking against 0.9.0, and found that amqp_simple_wait_frame_on_channel was missing from the shared lib. This could be a new problem.

[agriffis]

@jayvdb Sorry, three years ago might as well be an eternity for how much I remember about this.