Errors importing ExoCTK into a Miniconda environment

[JeffValenti]

I installed Miniconda with python 3.6 on a Mac:

I created a Miniconda environment named exoctk:

% cat environment.yml

name: exoctk
channels:
  - defaults
dependencies:
  - astropy
  - cython
  - python
  - bokeh
  - h5py
  - matplotlib
  - numba
  - scipy
  - pip:
    - astroquery
    - bibtexparser

% conda env create -f environment.yml

% conda info

     active environment : exoctk
    active env location : /Users/valenti/miniconda3/envs/exoctk
            shell level : 1
       user config file : /Users/valenti/.condarc
 populated config files : /Users/valenti/.condarc
          conda version : 4.5.4
    conda-build version : not installed
         python version : 3.6.5.final.0
       base environment : /Users/valenti/miniconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/osx-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/osx-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/osx-64
                          https://repo.anaconda.com/pkgs/pro/noarch
          package cache : /Users/valenti/miniconda3/pkgs
                          /Users/valenti/.conda/pkgs
       envs directories : /Users/valenti/miniconda3/envs
                          /Users/valenti/.conda/envs
               platform : osx-64
             user-agent : conda/4.5.4 requests/2.18.4 CPython/3.6.5 Darwin/16.7.0 OSX/10.12.6
                UID:GID : 3551:1031
             netrc file : None
           offline mode : False

I cloned master (d1f2efa2047be309727822274bedbe7c535aa3a4):

% git clone https://github.com/ExoCTK/ExoCTK.git

Install reports success:

% pip install .

Processing /Users/valenti/ExoCTK
...
Building wheels for collected packages: ExoCTK
  Running setup.py bdist_wheel for ExoCTK ... done
  Stored in directory: /private/var/folders/pf/yxxc6fzx7yxbymx008_rrj2w0003fz/T/pip-ephem-wheel-cache-_4rtzhka/wheels/89/59/38/1eaf5d5ac39f72c05f2d23a44a8612158071df43ecf5e72b7f
Successfully built ExoCTK
Installing collected packages: pysynphot, imagesize, snowballstemmer, docutils, babel, sphinxcontrib-websupport, Pygments, alabaster, sphinx, sphinx-automodapi, sphinx-rtd-theme, svo-filters, batman-package, ExoCTK
Successfully installed ExoCTK-0.1.3.dev743 Pygments-2.2.0 alabaster-0.7.10 babel-2.6.0 batman-package-2.4.6 docutils-0.14 imagesize-1.0.0 pysynphot-0.9.12 snowballstemmer-1.2.1 sphinx-1.7.5 sphinx-automodapi-0.7 sphinx-rtd-theme-0.4.0 sphinxcontrib-websupport-1.1.0 svo-filters-0.1.17

Importing ExoCTK prints a warning:

% python

>>> import ExoCTK

/Users/valenti/junk/ExoCTK/ExoCTK/contam_visibility/visibilityPA.py:18: UserWarning: 
This call to matplotlib.use() has no effect because the backend has already
been chosen; matplotlib.use() must be called *before* pylab, matplotlib.pyplot,
or matplotlib.backends is imported for the first time.

The backend was *originally* set to 'MacOSX' by the following code:
  File "<stdin>", line 1, in <module>
  File "/Users/valenti/junk/ExoCTK/ExoCTK/__init__.py", line 15, in <module>
    from .core import *
  File "/Users/valenti/junk/ExoCTK/ExoCTK/core.py", line 21, in <module>
    import matplotlib.pyplot as plt
  File "/Users/valenti/miniconda3/envs/junk/lib/python3.6/site-packages/matplotlib/pyplot.py", line 71, in <module>
    from matplotlib.backends import pylab_setup
  File "/Users/valenti/miniconda3/envs/junk/lib/python3.6/site-packages/matplotlib/backends/__init__.py", line 16, in <module>
    line for line in traceback.format_stack()
  matplotlib.use('Agg')

and later raises and error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/valenti/junk/ExoCTK/ExoCTK/__init__.py", line 16, in <module>
    from . import contam_visibility
  File "/Users/valenti/junk/ExoCTK/ExoCTK/contam_visibility/__init__.py", line 9, in <module>
    from . import sossContamFig
  File "/Users/valenti/junk/ExoCTK/ExoCTK/contam_visibility/sossContamFig.py", line 6, in <module>
    from bokeh.io import gridplot, show
ImportError: cannot import name 'gridplot'

[JeffValenti]

The warning is caused by line 18 of ExoCTK/ExoCTK/contam_visibility/visibilityPA.py:

matplotlib.use('Agg')

Since the warning says the statement has no effect, I commented it out to suppress the warning:

# matplotlib.use('Agg')

[JeffValenti]

The error is caused by line 6 of ExoCTK/ExoCTK/contam_visibility/sossContamFig.py:

from bokeh.io import gridplot, show

The issue is that gridplot is in bokeh.layouts, not bokeh.io. I replaced the line above with these two lines:

from bokeh.io import show
from bokeh.layouts import gridplot

These changes fixed the warning and the error described above.

[JeffValenti]

After the changes above, attempting to import ExoCTK raises a new issue:

>>> import ExoCTK
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/valenti/ExoCTK/ExoCTK/__init__.py", line 16, in <module>
    from . import contam_visibility
  File "/Users/valenti/ExoCTK/ExoCTK/contam_visibility/__init__.py", line 14, in <module>
    from . import sossFieldSim
  File "/Users/valenti/ExoCTK/ExoCTK/contam_visibility/sossFieldSim.py", line 9, in <module>
    import idlsave
ModuleNotFoundError: No module named 'idlsave'

The issue is that I didn't install idlsave in my conda environment. The idlsave repository says, "IDLSave has now been merged into Scipy, and can be used as scipy.io.readsav".

ExoCTK already requires scipy. Also, scipy is available via the default conda channel. Thus, I switched the code from idlsave to readsav. Here are the original lines of code:

import idlsave
...
modelParam = idlsave.read(os.path.join(idlsave_path,'modelsInfo.sav'),verbose=False)
...
modelO12 = idlsave.read(fNameModO12,verbose=False)['modelo12']

and here are the new lines of code:

from scipy.io import readsav
...
modelParam = readsav(os.path.join(idlsave_path,'modelsInfo.sav'),verbose=False)
...
modelO12 = readsav(fNameModO12,verbose=False)['modelo12']

These changes address the idlsave issue, which wasn't really an error.

[JeffValenti]

After the changes above, attempting to import ExoCTK raises a new error:

>>> import ExoCTK

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/valenti/junk/ExoCTK/ExoCTK/__init__.py", line 17, in <module>
    from . import forward_models
  File "/Users/valenti/junk/ExoCTK/ExoCTK/forward_models/__init__.py", line 5, in <module>
    from . import exotransmit
  File "/Users/valenti/junk/ExoCTK/ExoCTK/forward_models/exotransmit.py", line 2, in <module>
    from . import _exotransmit_wrapper
ImportError: cannot import name '_exotransmit_wrapper'

The problem is that the Cython code in ExoCTK/ExoCTK/forward_models/_exotransmit_wrapper.pyx is not getting cythonized into _exotransmit_wrapper.c.

I tracked the problem down to three instances of pal in setup files. We recently changed our package names from short abbreviations to longer descriptions. pal became forward_models.

The file ExoCTK/setup.cfg contained the lines:

[entry_points]
exotransmit = ExoCTK.pal._exotransmit_wrapper:exotransmit
collect_exotransmit_data = ExoCTK.pal.exotransmit:collect_exotransmit_data

which I changed to:

[entry_points]
exotransmit = ExoCTK.forward_models._exotransmit_wrapper:exotransmit
collect_exotransmit_data = ExoCTK.forward_models.exotransmit:collect_exotransmit_data

The file ExoCTK/ExoCTK/forward_models/setup_package.py contained the line:

    return [Extension(name='ExoCTK.pal._exotransmit_wrapper',

which I changed to:

    return [Extension(name='ExoCTK.forward_models._exotransmit_wrapper',

These changes alone don't seem to fix the error. Exiting and restarting python yields the same error. Using pip to uninstall and then install ExoCTK again yields the same error. Removing and recreating the conda environment yields the same error. Perhaps the earlier failed install left some improper files around that don't get cleaned up by pip uninstall ExoCTK.

[hover2pi]

@JeffValenti Try installing again. The Travis tests are all passing so this may be resolved. Thanks!