Pystan in Spyder - Githubissues

floklimm commented 7 years ago

Summary:

Running pystan in spyder leads to a Cython compiler crash.

Description:

Pystan initially runs fine in Spyder. However, if you run the same script a second time in the same console it leads to a Cython compiler crash. The error does not occur when just running two fits in the same script. So it seems to be a problem with executing the script a second time not the execution of pystan a second time per se.

Reproducible Steps:

Just running the '8 schools' example twice reproduces the error.

Current Output:

The error message is:

INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_286b3180dfa752c4cfedaf0241add0e4 NOW.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 699, in runfile
    execfile(filename, namespace)
  File "/usr/lib/python2.7/dist-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 81, in execfile
    builtins.execfile(filename, *where)
  File "/home/klimm/repos/netstan/pystan_error.py", line 15, in <module>
    fit1 = pystan.stan(file='8schools.stan', data=schools_dat, iter=1000, chains=4)
  File "/home/klimm/.local/lib/python2.7/site-packages/pystan/api.py", line 372, in stan
    verbose=verbose)
  File "/home/klimm/.local/lib/python2.7/site-packages/pystan/model.py", line 300, in __init__
    quiet=not verbose)
  File "/home/klimm/.local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 925, in cythonize
    cythonize_one(*args)
  File "/home/klimm/.local/lib/python2.7/site-packages/Cython/Build/Dependencies.py", line 1030, in cythonize_one
    result = compile([pyx_file], options)
  File "/home/klimm/.local/lib/python2.7/site-packages/Cython/Compiler/Main.py", line 687, in compile
    return compile_multiple(source, options)
  File "/home/klimm/.local/lib/python2.7/site-packages/Cython/Compiler/Main.py", line 665, in compile_multiple
    result = run_pipeline(source, options, context=context)
  File "/home/klimm/.local/lib/python2.7/site-packages/Cython/Compiler/Main.py", line 495, in run_pipeline
    err, enddata = Pipeline.run_pipeline(pipeline, source)
  File "/home/klimm/.local/lib/python2.7/site-packages/Cython/Compiler/Pipeline.py", line 340, in run_pipeline
    data = phase(data)
  File "Cython/Compiler/Visitor.py", line 286, in Cython.Compiler.Visitor.CythonTransform.__call__ (/tmp/pip-Uw4yYh-build/Cython/Compiler/Visitor.c:7414)
  File "Cython/Compiler/Visitor.py", line 269, in Cython.Compiler.Visitor.VisitorTransform.__call__ (/tmp/pip-Uw4yYh-build/Cython/Compiler/Visitor.c:7086)
  File "Cython/Compiler/Visitor.py", line 182, in Cython.Compiler.Visitor.TreeVisitor._visit (/tmp/pip-Uw4yYh-build/Cython/Compiler/Visitor.c:5330)
  File "Cython/Compiler/Visitor.py", line 180, in Cython.Compiler.Visitor.TreeVisitor._visit (/tmp/pip-Uw4yYh-build/Cython/Compiler/Visitor.c:5272)
  File "Cython/Compiler/Visitor.py", line 296, in Cython.Compiler.Visitor.CythonTransform.visit_Node (/tmp/pip-Uw4yYh-build/Cython/Compiler/Visitor.c:7620)
  File "Cython/Compiler/Visitor.py", line 247, in Cython.Compiler.Visitor.VisitorTransform.visitchildren (/tmp/pip-Uw4yYh-build/Cython/Compiler/Visitor.c:6405)
  File "Cython/Compiler/Visitor.py", line 247, in Cython.Compiler.Visitor.VisitorTransform.visitchildren (/tmp/pip-Uw4yYh-build/Cython/Compiler/Visitor.c:6837)
  File "Cython/Compiler/Visitor.py", line 248, in Cython.Compiler.Visitor.VisitorTransform.visitchildren (/tmp/pip-Uw4yYh-build/Cython/Compiler/Visitor.c:6425)
  File "Cython/Compiler/Visitor.py", line 221, in Cython.Compiler.Visitor.TreeVisitor._visitchildren (/tmp/pip-Uw4yYh-build/Cython/Compiler/Visitor.c:6108)
  File "Cython/Compiler/Visitor.py", line 193, in Cython.Compiler.Visitor.TreeVisitor._visitchild (/tmp/pip-Uw4yYh-build/Cython/Compiler/Visitor.c:5593)
  File "Cython/Compiler/Visitor.py", line 188, in Cython.Compiler.Visitor.TreeVisitor._visit (/tmp/pip-Uw4yYh-build/Cython/Compiler/Visitor.c:5474)
  File "Cython/Compiler/Visitor.py", line 147, in Cython.Compiler.Visitor.TreeVisitor._raise_compiler_error (/tmp/pip-Uw4yYh-build/Cython/Compiler/Visitor.c:4480)
Cython.Compiler.Errors.CompilerCrash: 
Error compiling Cython file:
------------------------------------------------------------
...
# file (after substitutions have been made).
#
#-----------------------------------------------------------------------------

# cython imports
from libcpp cimport bool
^
------------------------------------------------------------

/tmp/tmpvx9Pos/stanfit4anon_model_286b3180dfa752c4cfedaf0241add0e4_7126673893329254698.pyx:21:0: Compiler crash in RemoveUnreachableCode

ModuleNode.body = StatListNode(stanfit4anon_model_286b3180dfa752c4cfedaf0241add0e4_7126673893329254698.pyx:21:0)

Compiler crash traceback from this point on:
  File "Cython/Compiler/Visitor.py", line 180, in Cython.Compiler.Visitor.TreeVisitor._visit (/tmp/pip-Uw4yYh-build/Cython/Compiler/Visitor.c:5252)
    return handler_method(obj)
  File "/home/klimm/.local/lib/python2.7/site-packages/Cython/Compiler/ParseTreeTransforms.py", line 2253, in visit_StatListNode
    if not self.current_directives['remove_unreachable']:
TypeError: 'NoneType' object has no attribute '__getitem__'

Expected Output:

It should just run again. My current workaround is to open a new console, which works but is a bit annoying.

PyStan Version: 2.12.0.0

Python Version: 2.7.12

Spyder Version: 2.3.8

Operating System: Ubuntu 16.04.1 LTS

mhashemi0873 commented 7 years ago

I get error for running 8 schools example by pystan. I used: Python 2.7.12 :: Anaconda custom (64-bit) Ubuntu 16.04 LTS

I have tried all the installation guidelines (pip, github,...), with python 2.7 and 3.5, executing from /home, /tmp/,... but I get the following error:

runfile('/tmp/eightschools.py', wdir='/tmp') INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_95013624776d537c3cd7cd4d641c30e0 NOW. Traceback (most recent call last):

File "", line 1, in runfile('/tmp/eightschools.py', wdir='/tmp')

File "/home/meysam/anaconda2/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 866, in runfile execfile(filename, namespace)

File "/home/meysam/anaconda2/lib/python2.7/site-packages/spyder/utils/site/sitecustomize.py", line 94, in execfile builtins.execfile(filename, *where)

File "/tmp/eightschools.py", line 37, in iter=1000, chains=4)

File "/home/meysam/anaconda2/lib/python2.7/site-packages/pystan/api.py", line 368, in stan verbose=verbose)

File "/home/meysam/anaconda2/lib/python2.7/site-packages/pystan/model.py", line 315, in init self.module = load_module(module_name, lib_dir)

File "/home/meysam/anaconda2/lib/python2.7/site-packages/pystan/model.py", line 54, in load_module return imp.load_module(module_name, *module_info)

ImportError: /tmp/tmpLK2qQa/pystan/stanfit4anon_model_95013624776d537c3cd7cd4d641c30e0_724952c86beb130db392f6b01aa1ef05.so: undefined symbol: _ZNSt8ios_base7failureB5cxx11D1Ev

Interestingly, in Mac OS I don't get any error. Thanks a lot in advance for any help. Best, Meysam

mhashemi0873 commented 7 years ago

I tried with Jupyter and get ImportError:

ImportError Traceback (most recent call last)

in () 35 36 fit = pystan.stan(model_code=schools_code, data=schools_dat, ---> 37 iter=1000, chains=4) /home/meysam/anaconda2/lib/python2.7/site-packages/pystan/api.pyc in stan(file, model_name, model_code, fit, data, pars, chains, iter, warmup, thin, init, seed, algorithm, control, sample_file, diagnostic_file, verbose, boost_lib, eigen_lib, n_jobs, **kwargs) 366 m = StanModel(file=file, model_name=model_name, model_code=model_code, 367 boost_lib=boost_lib, eigen_lib=eigen_lib, --> 368 verbose=verbose) 369 # check that arguments in kwargs are valid 370 valid_args = {"chain_id", "init_r", "test_grad", "append_samples", "enable_random_init", /home/meysam/anaconda2/lib/python2.7/site-packages/pystan/model.pyc in __init__(self, file, charset, model_name, model_code, stanc_ret, boost_lib, eigen_lib, verbose, obfuscate_model_name) 313 os.dup2(orig_stderr, sys.stderr.fileno()) 314 --> 315 self.module = load_module(module_name, lib_dir) 316 self.fit_class = getattr(self.module, "StanFit4Model") 317 /home/meysam/anaconda2/lib/python2.7/site-packages/pystan/model.pyc in load_module(module_name, module_path) 52 # Python 2.7 53 module_info = imp.find_module(module_name, [module_path]) ---> 54 return imp.load_module(module_name, *module_info) 55 56 ImportError: /tmp/tmphN8u65/pystan/stanfit4anon_model_95013624776d537c3cd7cd4d641c30e0_724952c86beb130db392f6b01aa1ef05.so: undefined symbol: _ZNSt8ios_base7failureB5cxx11D1Ev

ariddell commented 7 years ago

This seems likely to be related to the new Cython version used in the most recent version of Anaconda. Thanks for the report.