Possible errors in CTF's generated setup.py

[huttered40]

On Porter machine, I notice that once the CTF library has been configured (with --build-scalapack) and built (with make and make python), the generated setup.py file appears to have a few issues.

From my experience, I do

python setup.py develop --user

with any python library to allow the tag (ctf in this case) to be known by the machine and thus able to serve as a dependency of other libraries.

Usually there are no issues, but running this command from the main directory does not work.

This is the initial generated setup.py:

from distutils.core import setup
from distutils.extension import Extension
import numpy

import os
os.environ["CXX"] = "mpicxx"
os.environ["CC"] = "mpicxx"

from Cython.Distutils import build_ext
from Cython.Build import cythonize

mod_core = ["ctf.core", "ctf/core.pyx"]
mod_rand = ["ctf.random", "ctf/random.pyx"]
mods = [mod_core, mod_rand]
ext_mods = []
for mod in mods:
    ext_mods.append(Extension(
            mod[0],
            [mod[1]],
            language="c++",
            include_dirs=[numpy.get_include(),"/home/hutter2/hutter2/ExternalLibraries/ctf/include","."],
            libraries="-lctf -Wl,-Bdynamic -lscalapack  -Wl,-Bdynamic -lblas -Wl,-Bdynamic -llapack".replace('-l','').replace('-Wl,-Bdynamic','').split(),
            extra_compile_args="-O3 -fopenmp ".split(),
            extra_link_args="-L/home/hutter2/hutter2/ExternalLibraries/ctf/lib_shared -O3 -fopenmp  -L/home/hutter2/hutter2/ExternalLibraries/ctf/scalapack/build/lib  -Wl,-rpath=/home/hutter2/hutter2/ExternalLibraries/ctf/scalapack/build/lib  ".split()
        ))

setup(name="CTF",packages=["ctf"],version="1.5.5",cmdclass = {'build_ext': build_ext},ext_modules = cythonize(ext_mods))

First thing I notice is that the mod_core and mod_rand paths seem wrong. I modified them in this new version by adding the "src_python/" prefix:

from distutils.core import setup
from distutils.extension import Extension
import numpy

import os
os.environ["CXX"] = "mpicxx"
os.environ["CC"] = "mpicxx"

from Cython.Distutils import build_ext
from Cython.Build import cythonize

mod_core = ["ctf.core", "src_python/ctf/core.pyx"]
mod_rand = ["ctf.random", "src_python/ctf/random.pyx"]
mods = [mod_core, mod_rand]
ext_mods = []
for mod in mods:
    ext_mods.append(Extension(
            mod[0],
            [mod[1]],
            language="c++",
            include_dirs=[numpy.get_include(),"/home/hutter2/hutter2/ExternalLibraries/ctf/include","."],
            libraries="-lctf -Wl,-Bdynamic -lscalapack  -Wl,-Bdynamic -lblas -Wl,-Bdynamic -llapack".replace('-l','').replace('-Wl,-Bdynamic','').split(),
            extra_compile_args="-O3 -fopenmp ".split(),
            extra_link_args="-L/home/hutter2/hutter2/ExternalLibraries/ctf/lib_shared -O3 -fopenmp  -L/home/hutter2/hutter2/ExternalLibraries/ctf/scalapack/build/lib  -Wl,-rpath=/home/hutter2/hutter2/ExternalLibraries/ctf/scalapack/build/lib  ".split()
        ))

setup(name="CTF",packages=["ctf"],version="1.5.5",cmdclass = {'build_ext': build_ext},ext_modules = cythonize(ext_mods))

Then, I notice that it does not understand the develop command. I replace from distutils.core import setup with from setuptools import setup to get the new setup.py:

from setuptools import setup
from distutils.extension import Extension
import numpy

import os
os.environ["CXX"] = "mpicxx"
os.environ["CC"] = "mpicxx"

from Cython.Distutils import build_ext
from Cython.Build import cythonize

mod_core = ["ctf.core", "src_python/ctf/core.pyx"]
mod_rand = ["ctf.random", "src_python/ctf/random.pyx"]
mods = [mod_core, mod_rand]
ext_mods = []
for mod in mods:
    ext_mods.append(Extension(
            mod[0],
            [mod[1]],
            language="c++",
            include_dirs=[numpy.get_include(),"/home/hutter2/hutter2/ExternalLibraries/ctf/include","."],
            libraries="-lctf -Wl,-Bdynamic -lscalapack  -Wl,-Bdynamic -lblas -Wl,-Bdynamic -llapack".replace('-l','').replace('-Wl,-Bdynamic','').split(),
            extra_compile_args="-O3 -fopenmp ".split(),
            extra_link_args="-L/home/hutter2/hutter2/ExternalLibraries/ctf/lib_shared -O3 -fopenmp  -L/home/hutter2/hutter2/ExternalLibraries/ctf/scalapack/build/lib  -Wl,-rpath=/home/hutter2/hutter2/ExternalLibraries/ctf/scalapack/build/lib  ".split()
        ))

setup(name="CTF",packages=["ctf"],version="1.5.5",cmdclass = {'build_ext': build_ext},ext_modules = cythonize(ext_mods))

Finally, I run the command python setup.py develop --user again and get the following output:

running develop
running egg_info
writing CTF.egg-info/PKG-INFO
writing top-level names to CTF.egg-info/top_level.txt
writing dependency_links to CTF.egg-info/dependency_links.txt
error: package directory 'ctf' does not exist

At this point, I am unsure as to why its giving an error.