Cannot import with latest pymc3 & theano

[jradavenport]

Describe the bug import starry throws a Theano error

To Reproduce import starry

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-53b69aeac1cd> in <module>
      1 import matplotlib.pyplot as plt
      2 import numpy as np
----> 3 import starry
      4 
      5 np.random.seed(12)

~/opt/anaconda3/lib/python3.8/site-packages/starry/__init__.py in <module>
     17 # Import the main interface
     18 from ._config import config
---> 19 from . import kepler, linalg, maps
     20 from .maps import Map
     21 from .kepler import Primary, Secondary, System

~/opt/anaconda3/lib/python3.8/site-packages/starry/kepler.py in <module>
      2 from . import config
      3 from ._constants import *
----> 4 from .maps import MapBase, RVBase, ReflectedBase
      5 from ._core import OpsSystem, math, linalg
      6 import numpy as np

~/opt/anaconda3/lib/python3.8/site-packages/starry/maps.py in <module>
      2 from . import config
      3 from ._constants import *
----> 4 from ._core import OpsYlm, OpsLD, OpsReflected, OpsRV, linalg, math
      5 from ._indices import integers, get_ylm_inds, get_ul_inds, get_ylmw_inds
      6 from ._plotting import (

~/opt/anaconda3/lib/python3.8/site-packages/starry/_core/__init__.py in <module>
      1 # -*- coding: utf-8 -*-
----> 2 from . import core, ops, utils
      3 
      4 from .core import *
      5 from .math import math, linalg

~/opt/anaconda3/lib/python3.8/site-packages/starry/_core/core.py in <module>
     31     # starry requires exoplanet >= v0.2.0
     32     from packaging import version
---> 33     import exoplanet
     34 
     35     if version.parse(exoplanet.__version__) < version.parse("0.2.0"):

~/opt/anaconda3/lib/python3.8/site-packages/exoplanet/__init__.py in <module>
     11 ]
     12 
---> 13 from . import distributions, interp, orbits
     14 from .citations import CITATIONS
     15 from .distributions import *  # NOQA

~/opt/anaconda3/lib/python3.8/site-packages/exoplanet/distributions/__init__.py in <module>
     14 ]
     15 
---> 16 from . import eccentricity
     17 from .base import Angle, Periodic, UnitDisk, UnitUniform, UnitVector
     18 from .deprecated import RadiusImpact, get_joint_radius_impact  # NOQA

~/opt/anaconda3/lib/python3.8/site-packages/exoplanet/distributions/eccentricity.py in <module>
      4 
      5 import numpy as np
----> 6 import pymc3 as pm
      7 import theano.tensor as tt
      8 

~/opt/anaconda3/lib/python3.8/site-packages/pymc3/__init__.py in <module>
     39 __set_compiler_flags()
     40 
---> 41 from . import gp, ode, sampling
     42 from .backends import load_trace, save_trace
     43 from .backends.tracetab import *

~/opt/anaconda3/lib/python3.8/site-packages/pymc3/gp/__init__.py in <module>
     14 
     15 from . import cov, mean, util
---> 16 from .gp import TP, Latent, LatentKron, Marginal, MarginalKron, MarginalSparse

~/opt/anaconda3/lib/python3.8/site-packages/pymc3/gp/gp.py in <module>
     23 import pymc3 as pm
     24 
---> 25 from pymc3.distributions import draw_values
     26 from pymc3.gp.cov import Constant, Covariance
     27 from pymc3.gp.mean import Zero

~/opt/anaconda3/lib/python3.8/site-packages/pymc3/distributions/__init__.py in <module>
     13 #   limitations under the License.
     14 
---> 15 from . import shape_utils, timeseries, transforms
     16 from .bart import BART
     17 from .bound import Bound

~/opt/anaconda3/lib/python3.8/site-packages/pymc3/distributions/timeseries.py in <module>
     21 from theano import scan
     22 
---> 23 from . import distribution, multivariate
     24 from .continuous import Flat, Normal, get_tau_sigma
     25 from .shape_utils import to_tuple

~/opt/anaconda3/lib/python3.8/site-packages/pymc3/distributions/multivariate.py in <module>
     25 from scipy import linalg, stats
     26 from theano.gof.op import get_test_value
---> 27 from theano.gof.utils import TestValueError
     28 from theano.tensor.nlinalg import det, eigh, matrix_inverse, trace
     29 from theano.tensor.slinalg import Cholesky

ImportError: cannot import name 'TestValueError' from 'theano.gof.utils' (/Users/james/opt/anaconda3/lib/python3.8/site-packages/theano/gof/utils.py)

Your setup (please complete the following information):

• Version of starry: 1.0.0 installed via pip.
• Operating system: macOS 11.1
• Python version & installation method (pip, conda, etc.): Python 3.8.5 via anaconda.

exoplanet also installed via pip and has same error

[jradavenport]

also the version of theano is 1.0.5, and I cannot import pymc3, so I think that's where things are broken?

[dfm]

This is not really a starry problem. The most recent versions of PyMC3 and Theano have tons of compatibility issues and you're getting a collision between Theano and Theano-PyMC. I'd recommend doing a clean install with PyMC3<3.10 and Theano==1.0.5 (which will work and is probably not what you actually have installed even if it seems that way!).

[jradavenport]

OK, I tried to do a clean install w/ pip install pymc3==3.8.0, and then pip install theano==1.0.5.

It's definitely not a starry/exoplanet problem, b/c I still can't even import pymc3 or theano right now on the M1 MacBook Pro. Hmm. Time to do some reading on those package's issues!

[dfm]

Uh oh! What error do you get now?

[jradavenport]

In [1]: import pymc3
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-200e74b8195d> in <module>
----> 1 import pymc3

~/opt/anaconda3/lib/python3.8/site-packages/pymc3/__init__.py in <module>
      3 
      4 from .blocking import *
----> 5 from .distributions import *
      6 from .distributions import transforms
      7 from .glm import *

~/opt/anaconda3/lib/python3.8/site-packages/pymc3/distributions/__init__.py in <module>
----> 1 from . import timeseries
      2 from . import transforms
      3 from . import shape_utils
      4 
      5 from .continuous import Uniform

~/opt/anaconda3/lib/python3.8/site-packages/pymc3/distributions/timeseries.py in <module>
      1 from scipy import stats
----> 2 import theano.tensor as tt
      3 from theano import scan
      4 
      5 from pymc3.util import get_variable_name

~/opt/anaconda3/lib/python3.8/site-packages/theano/__init__.py in <module>
    108     object2, utils)
    109 
--> 110 from theano.compile import (
    111     SymbolicInput, In,
    112     SymbolicOutput, Out,

~/opt/anaconda3/lib/python3.8/site-packages/theano/compile/__init__.py in <module>
     26 from theano.compile.builders import *
     27 
---> 28 from theano.compile.function import function, function_dump

~/opt/anaconda3/lib/python3.8/site-packages/theano/compile/function/__init__.py in <module>
      5 from collections import OrderedDict
      6 
----> 7 from theano.compile.function.pfunc import pfunc
      8 from theano.compile.function.types import orig_function
      9 

~/opt/anaconda3/lib/python3.8/site-packages/theano/compile/function/pfunc.py in <module>
      8 
      9 from theano import config
---> 10 from theano.compile.function.types import UnusedInputError, orig_function
     11 from theano.compile.io import In, Out
     12 from theano.compile.profiling import ProfileStats

~/opt/anaconda3/lib/python3.8/site-packages/theano/compile/function/types.py in <module>
     21 from theano.gof import graph
     22 from theano.gof.op import ops_with_inner_function
---> 23 from theano.gof.toolbox import is_same_graph
     24 
     25 

ImportError: cannot import name 'is_same_graph' from 'theano.gof.toolbox' (/Users/james/opt/anaconda3/lib/python3.8/site-packages/theano/gof/toolbox.py)

[dfm]

This suggests to me that you still have theano-pymc installed. Can you tell me exactly which steps you went through to install?

(This isn't anything to do with the M1 - it's issues with colliding namespaces!)

[lgrcia]

I get the same issue at pymc3 import when both exoplanet and starry are installed through pip within a virtual env:

• python 3.6.5
• starry==1.0.0
• exoplanet==0.4.3
• tested on macOS 10.14.6 and macOS 10.15.6

Same setup(s) with python 3.7.5 gives no error when importing pymc3. However when importig exoplanet I get

[...]/exoplanet/theano_ops/contact_points.py", line 14, in <module> class ContactPoints(theano.Op):
AttributeError: module 'theano' has no attribute 'Op'

I couldn't find this issue in the exoplanet existing ones

[rodluger]

Interesting. I'm planning a new release of starry that should help with these compatibility issues. The particular error you're getting is within exoplanet. @dfm, any ideas?

[dfm]

This is the same problem: @lgrcia, you must have Theano-pymc installed. Starry is not compatible with (several!) of the most recent versions of PyMC3 or the Theano-pymc project. You'll need to run

pip uninstall theano-pymc  # run a few times until it says not installed
pip install "pymc3<3.10" "theano==1.0.5"

[rodluger]

@dfm Do you think I should try to upgrade stuff in starry to rely on theano-pymc? Or is it still too volatile?

[lgrcia]

I notice that pip install starry is collecting theano-pymc so I will have to uninstall it every time I install starry. Ignore if that doesn't help. It is solved and usable on my side. Many thanks

[dfm]

@lgrcia: that's because starry doesn't pin PyMC3. If you manually install PyMC3 and theano first then pip install starry shouldn't try to update.

@rodluger: I think it's worth trying to keep it up to date but they're breaking stuff pretty fast over there so we won't always be able to stay up to date. For example, the current exoplanet is compatible with the last minor version of theano-pymc but the current release looks like it won't even import!

[rodluger]

Ugh. Sounds good. I'll aim for a new release of starry next week (ish). Let's chat more offline about this.

[dfm]

This did the trick for exoplanet: https://github.com/exoplanet-dev/exoplanet/blob/testval/src/exoplanet/theano_ops/compat.py

[edterrell]

I know this is closed, but after struggling to get pymc3 worining on an imac, I stumbled across the following which worked for me on stackoverflow: https://stackoverflow.com/questions/65543096/i-cannot-import-pymc3

conda remove theano pip uninstall Theano Theano-PyMC PyMC3 pip install PyMC3