Building wheel for batman-package & starry

[darinadaly]

MacOS Catalina (10.15.4) Python 3.8.2 pip 20.1.1 (Not sure if this is actually a bug, but I accidentally labeled it and don't know how to remove it now)

I ran (in bash)

git clone https://github.com/exoplanet-dev/exoplanet.git
cd exoplanet
python -m pip install -e .
python -m pip install .[test]

and came across the error:

ERROR: Failed building wheel for starry
  Building wheel for batman-package (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/darinadaly/miniconda3/envs/project2/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/k6/2hd432893sd6_hjfyzr6v6440000gn/T/pip-install-bveqp_u5/batman-package/setup.py'"'"'; __file__='"'"'/private/var/folders/k6/2hd432893sd6_hjfyzr6v6440000gn/T/pip-install-bveqp_u5/batman-package/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/k6/2hd432893sd6_hjfyzr6v6440000gn/T/pip-wheel-j1k3l3s2
       cwd: /private/var/folders/k6/2hd432893sd6_hjfyzr6v6440000gn/T/pip-install-bveqp_u5/batman-package/
  Complete output (29 lines):
  Checking for OpenMP support...
  Did not detect OpenMP support.
  /Users/darinadaly/miniconda3/envs/project2/lib/python3.8/distutils/dist.py:274: UserWarning: Unknown distribution option: 'extras_requires'
    warnings.warn(msg)
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.9-x86_64-3.8
  creating build/lib.macosx-10.9-x86_64-3.8/batman
  copying batman/openmp.py -> build/lib.macosx-10.9-x86_64-3.8/batman
  copying batman/transitmodel.py -> build/lib.macosx-10.9-x86_64-3.8/batman
  copying batman/__init__.py -> build/lib.macosx-10.9-x86_64-3.8/batman
  copying batman/plots.py -> build/lib.macosx-10.9-x86_64-3.8/batman
  copying batman/tests.py -> build/lib.macosx-10.9-x86_64-3.8/batman
  running build_ext
  building 'batman._nonlinear_ld' extension
  creating build/temp.macosx-10.9-x86_64-3.8
  creating build/temp.macosx-10.9-x86_64-3.8/c_src
  x86_64-apple-darwin13.4.0-clang -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O3 -Wall -Wstrict-prototypes -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O3 -pipe -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=/Users/darinadaly/miniconda3/envs/project2=/usr/local/src/conda-prefix -flto -Wl,-export_dynamic -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O3 -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/darinadaly/miniconda3/envs/project2/include -D_FORTIFY_SOURCE=2 -mmacosx-version-min=10.9 -isystem /Users/darinadaly/miniconda3/envs/project2/include -I/Users/darinadaly/miniconda3/envs/project2/lib/python3.8/site-packages/numpy/core/include -I/Users/darinadaly/miniconda3/envs/project2/include/python3.8 -c c_src/_nonlinear_ld.c -o build/temp.macosx-10.9-x86_64-3.8/c_src/_nonlinear_ld.o -std=c99
  clang-10: warning: -Wl,-export_dynamic: 'linker' input unused [-Wunused-command-line-argument]
  c_src/_nonlinear_ld.c:67:10: warning: unknown pragma ignored [-Wunknown-pragmas]
          #pragma acc data copyin(intensity_args)
                  ^
  1 warning generated.
  x86_64-apple-darwin13.4.0-clang -bundle -undefined dynamic_lookup -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,/Users/darinadaly/miniconda3/envs/project2/lib -L/Users/darinadaly/miniconda3/envs/project2/lib -flto -Wl,-export_dynamic -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,/Users/darinadaly/miniconda3/envs/project2/lib -L/Users/darinadaly/miniconda3/envs/project2/lib -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,/Users/darinadaly/miniconda3/envs/project2/lib -L/Users/darinadaly/miniconda3/envs/project2/lib -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/darinadaly/miniconda3/envs/project2/include -D_FORTIFY_SOURCE=2 -mmacosx-version-min=10.9 -isystem /Users/darinadaly/miniconda3/envs/project2/include -arch x86_64 build/temp.macosx-10.9-x86_64-3.8/c_src/_nonlinear_ld.o -o build/lib.macosx-10.9-x86_64-3.8/batman/_nonlinear_ld.cpython-38-darwin.so
  ld: unknown option: -platform_version
  clang-10: error: linker command failed with exit code 1 (use -v to see invocation)
  error: command 'x86_64-apple-darwin13.4.0-clang' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for batman-package
  Running setup.py clean for batman-package
Successfully built exoplanet
Failed to build rebound starry batman-package
ERROR: Could not build wheels for starry which use PEP 517 and cannot be installed directly

I looked at issue #72 and tried pip install rebound

but was stopped at ERROR: Command errored out with exit status 1: /Users/darinadaly/miniconda3/envs/project2/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/k6/2hd432893sd6_hjfyzr6v6440000gn/T/pip-install-nz7vzsez/rebound/setup.py'"'"'; __file__='"'"'/private/var/folders/k6/2hd432893sd6_hjfyzr6v6440000gn/T/pip-install-nz7vzsez/rebound/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/k6/2hd432893sd6_hjfyzr6v6440000gn/T/pip-record-94_ct0i2/install-record.txt --single-version-externally-managed --compile --install-headers /Users/darinadaly/miniconda3/envs/project2/include/python3.8/rebound Check the logs for full command output.

I also tried running pip install --no-deps exoplanet but it didn't fix anything.

If I just run python -m pytest -vx tests, then tests/light_curves_test.py::test_light_curve_grad (and other tests) fails and I get the error:

ERROR: Command errored out with exit status 1: /Users/darinadaly/miniconda3/envs/project2/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/k6/2hd432893sd6_hjfyzr6v6440000gn/T/pip-install-nz7vzsez/rebound/setup.py'"'"'; __file__='"'"'/private/var/folders/k6/2hd432893sd6_hjfyzr6v6440000gn/T/pip-install-nz7vzsez/rebound/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/k6/2hd432893sd6_hjfyzr6v6440000gn/T/pip-record-94_ct0i2/install-record.txt --single-version-externally-managed --compile --install-headers /Users/darinadaly/miniconda3/envs/project2/include/python3.8/rebound Check the logs for full command output.

[dfm]

Unfortunately this doesn't seem to be an exoplanet issue so I'm not sure how much help I can be, but I'll try!

First, a few comments:

• pip install -e .[test] isn't testing the installation. It's installing the dependencies that are required to run the tests
• batman, starry, and rebound are not required to use exoplanet (they're only used to run tests), and they're separate packages. So the fact that you can't install either of those means that there's something wrong with your Python environment, not that there's a bug in exoplanet.
• Since the failures are happening for the packages that require compilation, this will also lead to problems running exoplanet.

To deal with compilation issues like this, I generally recommend trying to setup a fresh conda environment following the steps given in the docs. If that doesn't work, you should try looking into the errors being produced when compiling batman, starry, and rebound because those won't be specific to any of these packages.

[darinadaly]

Thank you for the suggestion! My apologies for misunderstanding, I am very new to coding. Actually my main issue now seems to be with theano when I run python -m pytest -vx tests/light_curves_test.py :

=================================== FAILURES ===================================
____________________________ test_light_curve_grad _____________________________

caplog = <_pytest.logging.LogCaptureFixture object at 0x1263b9eb0>

    def test_light_curve_grad(caplog):
        u_val = np.array([0.2, 0.3, 0.1, 0.5])
        b_val = np.linspace(-1.5, 1.5, 20)
        r_val = 0.1 + np.zeros_like(b_val)

        lc = lambda u, b, r: LimbDarkLightCurve(u)._compute_light_curve(  # NOQA
            b, r
        )

        with caplog.at_level(logging.DEBUG, logger="theano.gof.cmodule"):
>           utt.verify_grad(lc, [u_val, b_val, r_val])

tests/light_curves_test.py:61: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/tests/unittest_tools.py:92: in verify_grad
    T.verify_grad(op, pt, n_tests, rng, *args, **kwargs)
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/gradient.py:1772: in verify_grad
    grad_fn = function(tensor_pt, symbolic_grad,
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/gradient.py:1716: in function
    f = compile.function(inputs, output, accept_inplace=True,
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/compile/function.py:306: in function
    fn = pfunc(params=inputs,
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/compile/pfunc.py:483: in pfunc
    return orig_function(inputs, cloned_outputs, mode,
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/compile/function_module.py:1841: in orig_function
    fn = m.create(defaults)
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/compile/function_module.py:1714: in create
    _fn, _i, _o = self.linker.make_thunk(
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/gof/link.py:697: in make_thunk
    return self.make_all(input_storage=input_storage,
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/gof/vm.py:1087: in make_all
    thunks.append(node.op.make_thunk(node,
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/gof/op.py:954: in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map,
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/gof/op.py:857: in make_c_thunk
    outputs = cl.make_thunk(input_storage=node_input_storage,
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/gof/cc.py:1215: in make_thunk
    cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/gof/cc.py:1153: in __compile__
    thunk, module = self.cthunk_factory(error_storage,
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/gof/cc.py:1623: in cthunk_factory
    module = get_module_cache().module_from_key(
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/gof/cmodule.py:1189: in module_from_key
    module = lnk.compile_cmodule(location)
../miniconda3/envs/project2/lib/python3.8/site-packages/theano/gof/cc.py:1520: in compile_cmodule
    module = c_compiler.compile_str(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

module_name = 'm665f0931491ce050f501ea628e4c934d97809b6476ed91b82c0f7d4354b3598a'
src_code = '#include <Python.h>\n#include <iostream>\n#include "theano_mod_helper.h"\n#include <math.h>\n#include <numpy/arrayobj...ed91b82c0f7d4354b3598a(void) {\n   import_array();\n    PyObject *m = PyModule_Create(&moduledef);\n    return m;\n}\n'
location = '/Users/darinadaly/.theano/compiledir_macOS-10.15.4-x86_64-i386-64bit-i386-3.8.2-64/tmp0hztfhbh'
include_dirs = ['/Users/darinadaly/miniconda3/envs/project2/lib/python3.8/site-packages/numpy/core/include', '/Users/darinadaly/minic...roject2/include/python3.8', '/Users/darinadaly/miniconda3/envs/project2/lib/python3.8/site-packages/theano/gof/c_code']
lib_dirs = ['/Users/darinadaly/miniconda3/envs/project2/lib'], libs = []
preargs = ['-O3', '-fno-math-errno', '-Wno-unused-label', '-Wno-unused-variable', '-Wno-write-strings', '-march=skylake', ...]
py_module = True, hide_symbols = True

    @staticmethod
    def compile_str(module_name, src_code, location=None,
                    include_dirs=None, lib_dirs=None, libs=None,
                    preargs=None, py_module=True, hide_symbols=True):
        """
        Parameters
        ----------
        module_name : str
            This has been embedded in the src_code.
        src_code
            A complete c or c++ source listing for the module.
        location
            A pre-existing filesystem directory where the cpp file and .so will
            be written.
        include_dirs
            A list of include directory names (each gets prefixed with -I).
        lib_dirs
            A list of library search path directory names (each gets prefixed
            with -L).
        libs
            A list of libraries to link with (each gets prefixed with -l).
        preargs
            A list of extra compiler arguments.
        py_module
            If False, compile to a shared library, but do not import it as a
            Python module.
        hide_symbols
            If True (the default) all symbols will be hidden from the library
            symbol table (which means that other objects can't use them).

        Returns
        -------
        object
            Dynamically-imported python module of the compiled code (unless
            py_module is False, in that case returns None).

        """
        # TODO: Do not do the dlimport in this function

        if not theano.config.cxx:
            raise MissingGXX("g++ not available! We can't compile c code.")

        if include_dirs is None:
            include_dirs = []
        if lib_dirs is None:
            lib_dirs = []
        if libs is None:
            libs = []
        if preargs is None:
            preargs = []

        # Remove empty string directory
        include_dirs = [d for d in include_dirs if d]
        lib_dirs = [d for d in lib_dirs if d]

        include_dirs = include_dirs + std_include_dirs()
        libs = libs + std_libs()
        lib_dirs = lib_dirs + std_lib_dirs()

        cppfilename = os.path.join(location, 'mod.cpp')
        with open(cppfilename, 'w') as cppfile:

            _logger.debug('Writing module C++ code to %s', cppfilename)

            cppfile.write(src_code)
            # Avoid gcc warning "no newline at end of file".
            if not src_code.endswith('\n'):
                cppfile.write('\n')

        if platform.python_implementation() == 'PyPy':
            suffix = '.' + get_lib_extension()

            dist_suffix = distutils.sysconfig.get_config_var("SO")
            if dist_suffix is not None and dist_suffix != '':
                suffix = dist_suffix

            filepath = '%s%s' % (module_name, suffix)
        else:
            filepath = '%s.%s' % (module_name, get_lib_extension())

        lib_filename = os.path.join(location, filepath)

        _logger.debug('Generating shared lib %s', lib_filename)
        cmd = [theano.config.cxx, get_gcc_shared_library_arg(), '-g']

        if config.cmodule.remove_gxx_opt:
            cmd.extend(p for p in preargs if not p.startswith('-O'))
        else:
            cmd.extend(preargs)
        # to support path that includes spaces, we need to wrap it with double quotes on Windows
        path_wrapper = "\"" if os.name == 'nt' else ""
        cmd.extend(['-I%s%s%s' % (path_wrapper, idir, path_wrapper) for idir in include_dirs])
        cmd.extend(['-L%s%s%s' % (path_wrapper, ldir, path_wrapper) for ldir in lib_dirs])
        if hide_symbols and sys.platform != 'win32':
            # This has been available since gcc 4.0 so we suppose it
            # is always available. We pass it here since it
            # significantly reduces the size of the symbol table for
            # the objects we want to share. This in turns leads to
            # improved loading times on most platforms (win32 is
            # different, as usual).
            cmd.append('-fvisibility=hidden')
        cmd.extend(['-o', '%s%s%s' % (path_wrapper, lib_filename, path_wrapper)])
        cmd.append('%s%s%s' % (path_wrapper, cppfilename, path_wrapper))
        cmd.extend(['-l%s' % l for l in libs])
        # print >> sys.stderr, 'COMPILING W CMD', cmd
        _logger.debug('Running cmd: %s', ' '.join(cmd))

        def print_command_line_error():
            # Print command line when a problem occurred.
            print(("Problem occurred during compilation with the "
                   "command line below:"), file=sys.stderr)
            print(' '.join(cmd), file=sys.stderr)

        try:
            p_out = output_subprocess_Popen(cmd)
            compile_stderr = decode(p_out[1])
        except Exception:
            # An exception can occur e.g. if `g++` is not found.
            print_command_line_error()
            raise

        status = p_out[2]

        if status:
            tf = tempfile.NamedTemporaryFile(
                mode='w',
                prefix='theano_compilation_error_',
                delete=False
            )
            # gcc put its messages to stderr, so we add ours now
            tf.write('===============================\n')
            for i, l in enumerate(src_code.split('\n')):
                tf.write('%05i\t%s\n' % (i + 1, l))
            tf.write('===============================\n')
            tf.write("Problem occurred during compilation with the "
                     "command line below:\n")
            tf.write(' '.join(cmd))
            # Print errors just below the command line.
            tf.write(compile_stderr)
            tf.close()
            print('\nYou can find the C code in this temporary file: ' + tf.name)
            not_found_libraries = re.findall('-l["."-_a-zA-Z0-9]*', compile_stderr)
            for nf_lib in not_found_libraries:
                print('library ' + nf_lib[2:] + ' is not found.')
                if re.search('-lPYTHON["."0-9]*', nf_lib, re.IGNORECASE):
                    py_string = re.search('-lpython["."0-9]*', nf_lib, re.IGNORECASE).group()[8:]
                    if py_string != '':
                        print(
                            'Check if package python-dev ' + py_string + ' or python-devel ' + py_string + ' is installed.'
                        )
                    else:
                        print(
                            'Check if package python-dev or python-devel is installed.'
                        )

            # We replace '\n' by '. ' in the error message because when Python
            # prints the exception, having '\n' in the text makes it more
            # difficult to read.
>           raise Exception('Compilation failed (return status=%s): %s' %
E           

---

Exception: ('The following error happened while compiling the node', Elemwise{Mul}[(0, 1)](InplaceDimShuffle{x,0}.0, Reshape{2}.0), '\n', "Compilation failed (return status=1): /Users/darinadaly/.theano/compiledir_macOS-10.15.4-x86_64-i386-64bit-i386-3.8.2-64/tmp0hztfhbh/mod.cpp:463:27: error: non-constant-expression cannot be narrowed from type 'npy_intp' (aka 'long') to 'int' in initializer list [-Wc++11-narrowing]. int init_totals[2] = {V5_n0, V3_n1};. ^~~. /Users/darinadaly/.theano/compiledir_macOS-10.15.4-x86_64-i386-64bit-i386-3.8.2-64/tmp0hztfhbh/mod.cpp:463:27: note: insert an explicit cast to silence this issue. int init_totals[2] = {V5_n0, V3_n1};. ^. static_cast( ). /Users/darinadaly/.theano/compiledir_macOS-10.15.4-x86_64-i386-64bit-i386-3.8.2-64/tmp0hztfhbh/mod.cpp:463:34: error: non-constant-expression cannot be narrowed from type 'npy_intp' (aka 'long') to 'int' in initializer list [-Wc++11-narrowing]. int init_totals[2] = {V5_n0, V3_n1};. ^. /Users/darinadaly/.theano/compiledir_macOS-10.15.4-x86_64-i386-64bit-i386-3.8.2-64/tmp0hztfhbh/mod.cpp:463:34: note: insert an explicit cast to silence this issue. int init_totals[2] = {V5_n0, V3_n1};. ^. static_cast( ). /Users/darinadaly/.theano/compiledir_macOS-10.15.4-x86_64-i386-64bit-i386-3.8.2-64/tmp0hztfhbh/mod.cpp:475:12: error: non-constant-expression cannot be narrowed from type 'ssize_t' (aka 'long') to 'int' in initializer list [-Wc++11-narrowing]. 0, V3_stride1, . ^~. /Users/darinadaly/.theano/compiledir_macOS-10.15.4-x86_64-i386-64bit-i386-3.8.2-64/tmp0hztfhbh/mod.cpp:475:12: note: insert an explicit cast to silence this issue. 0, V3_stride1, . ^~. static_cast( ). /Users/darinadaly/.theano/compiledir_macOS-10.15.4-x86_64-i386-64bit-i386-3.8.2-64/tmp0hztfhbh/mod.cpp:476:1: error: non-constant-expression cannot be narrowed from type 'ssize_t' (aka 'long') to 'int' in initializer list [-Wc++11-narrowing]. V5_stride0, V5_stride1. ^~. /Users/darinadaly/.theano/compiledir_macOS-10.15.4-x86_64-i386-64bit-i386-3.8.2-64/tmp0hztfhbh/mod.cpp:476:1: note: insert an explicit cast to silence this issue. V5_stride0, V5_stride1. ^~. static_cast( ). /Users/darinadaly/.theano/compiledir_macOS-10.15.4-x86_64-i386-64bit-i386-3.8.2-64/tmp0hztfhbh/mod.cpp:476:13: error: non-constant-expression cannot be narrowed from type 'ssize_t' (aka 'long') to 'int' in initializer list [-Wc++11-narrowing]. V5_stride0, V5_stride1. ^~. /Users/darinadaly/.theano/compiledir_macOS-10.15.4-x86_64-i386-64bit-i386-3.8.2-64/tmp0hztfhbh/mod.cpp:476:13: note: insert an explicit cast to silence this issue. V5_stride0, V5_stride1. ^~~~. static_cast( ). 5 errors generated.. ", '[Elemwise{Mul}[(0, 1)](<TensorType(float64, row)>, <TensorType(float64, matrix)>)]')

---

[dfm]

No worries! It looks like you need to upgrade PyMC3 or add the following before you import exoplanet:

import theano.config
theano.config.gcc.cxxflags = "-Wno-c++11-narrowing"

Ref: https://github.com/pymc-devs/pymc3/issues/3695

[darinadaly]

This worked! Thank you so much!