pymc-devs / pymc

Bayesian Modeling and Probabilistic Programming in Python
https://docs.pymc.io/
Other
8.69k stars 2.01k forks source link

macOS: code works in jupyter/ipython but not in terminal #3872

Closed fohria closed 4 years ago

fohria commented 4 years ago

bayesian newbie here, thanks for this nice library!

Description of your problem

I've no idea what's going on here, but since there are several related issues, i thought it might be something weird going on that's macOS specific?

potentially related to:

3831

3695

3844

Please provide a minimal, self-contained, and reproducible example. sorry about the not-very-minimal example. main model definition is below, it uses theano scan and full code in this gist

actions = theano.shared(np.array([0,1,0,1,1,0,0], dtype='int16'))
rewards = theano.shared(np.array([4,2,6,3,2,5,4], dtype='int16'))

with pm.Model() as qlearn3:

    alpha = pm.Beta('alpha', alpha=1, beta=1)
    tau = pm.HalfNormal('tau', 10)
    gamma = pm.Beta('gamma', alpha=1, beta=1)

    probs = categorical_actionsQL(actions, rewards, alpha, tau, gamma)

    like = pm.Categorical('like', p=probs, observed=actions)

    trace = pm.sample()

Please provide the full traceback.

exhibit A

You can find the C code in this temporary file: /var/folders/kj/ztrcmdjs58ncsdg37pg5pynw0000gn/T/theano_compilation_error_7sn1f20a
Traceback (most recent call last):
  File "model.py", line 41, in <module>
    probs = categorical_actionsQL(actions, rewards, alpha, tau, gamma)
  File "model.py", line 23, in categorical_actionsQL
    output, updates = theano.scan(fn=update_qvalsQL,
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/scan_module/scan.py", line 1077, in scan
    scan_outs = local_op(*scan_inputs)
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/gof/op.py", line 669, in __call__
    thunk = node.op.make_thunk(node, storage_map, compute_map,
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/scan_module/scan_op.py", line 861, in make_thunk
    self.fn = function(wrapped_inputs,
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/compile/function.py", line 306, in function
    fn = pfunc(params=inputs,
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/compile/pfunc.py", line 483, in pfunc
    return orig_function(inputs, cloned_outputs, mode,
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/compile/function_module.py", line 1841, in orig_function
    fn = m.create(defaults)
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/compile/function_module.py", line 1714, in create
    _fn, _i, _o = self.linker.make_thunk(
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/gof/link.py", line 697, in make_thunk
    return self.make_all(input_storage=input_storage,
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/gof/vm.py", line 1087, in make_all
    thunks.append(node.op.make_thunk(node,
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/gof/op.py", line 954, in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map,
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/gof/op.py", line 857, in make_c_thunk
    outputs = cl.make_thunk(input_storage=node_input_storage,
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/gof/cc.py", line 1215, in make_thunk
    cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/gof/cc.py", line 1153, in __compile__
    thunk, module = self.cthunk_factory(error_storage,
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/gof/cc.py", line 1623, in cthunk_factory
    module = get_module_cache().module_from_key(
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/gof/cmodule.py", line 1189, in module_from_key
    module = lnk.compile_cmodule(location)
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/gof/cc.py", line 1520, in compile_cmodule
    module = c_compiler.compile_str(
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/theano/gof/cmodule.py", line 2410, in compile_str
    raise Exception('Compilation failed (return status=%s): %s' %
Exception: ('The following error happened while compiling the node', Elemwise{mul,no_inplace}(InplaceDimShuffle{x,0}.0, InplaceDimShuffle{x,x}.0), '\n', "Compilation failed (return status=1): /Users/foh/.theano/compiledir_macOS-10.14.6-x86_64-i386-64bit-i386-3.8.2-64/tmpzsu2aa1s/mod.cpp:506:27: error: non-constant-expression cannot be narrowed from type 'npy_intp' (aka 'long') to 'int' in initializer list [-Wc++11-narrowing].     int init_totals[2] = {V1_n0, V3_n1};.                           ^~~~~. /Users/foh/.theano/compiledir_macOS-10.14.6-x86_64-i386-64bit-i386-3.8.2-64/tmpzsu2aa1s/mod.cpp:506:27: note: insert an explicit cast to silence this issue.     int init_totals[2] = {V1_n0, V3_n1};.                           ^~~~~.                           static_cast<int>( ). /Users/foh/.theano/compiledir_macOS-10.14.6-x86_64-i386-64bit-i386-3.8.2-64/tmpzsu2aa1s/mod.cpp:506:34: error: non-constant-expression cannot be narrowed from type 'npy_intp' (aka 'long') to 'int' in initializer list [-Wc++11-narrowing].     int init_totals[2] = {V1_n0, V3_n1};.                                  ^~~~~. /Users/foh/.theano/compiledir_macOS-10.14.6-x86_64-i386-64bit-i386-3.8.2-64/tmpzsu2aa1s/mod.cpp:506:34: note: insert an explicit cast to silence this issue.     int init_totals[2] = {V1_n0, V3_n1};.                                  ^~~~~.                                  static_cast<int>( ). /Users/foh/.theano/compiledir_macOS-10.14.6-x86_64-i386-64bit-i386-3.8.2-64/tmpzsu2aa1s/mod.cpp:518:12: error: non-constant-expression cannot be narrowed from type 'ssize_t' (aka 'long') to 'int' in initializer list [-Wc++11-narrowing].         0, V3_stride1, .            ^~~~~~~~~~. /Users/foh/.theano/compiledir_macOS-10.14.6-x86_64-i386-64bit-i386-3.8.2-64/tmpzsu2aa1s/mod.cpp:518:12: note: insert an explicit cast to silence this issue.         0, V3_stride1, .            ^~~~~~~~~~.            static_cast<int>( ). /Users/foh/.theano/compiledir_macOS-10.14.6-x86_64-i386-64bit-i386-3.8.2-64/tmpzsu2aa1s/mod.cpp:520:1: error: non-constant-expression cannot be narrowed from type 'ssize_t' (aka 'long') to 'int' in initializer list [-Wc++11-narrowing]. V1_stride0, V1_stride1. ^~~~~~~~~~. /Users/foh/.theano/compiledir_macOS-10.14.6-x86_64-i386-64bit-i386-3.8.2-64/tmpzsu2aa1s/mod.cpp:520:1: note: insert an explicit cast to silence this issue. V1_stride0, V1_stride1. ^~~~~~~~~~. static_cast<int>( ). /Users/foh/.theano/compiledir_macOS-10.14.6-x86_64-i386-64bit-i386-3.8.2-64/tmpzsu2aa1s/mod.cpp:520:13: error: non-constant-expression cannot be narrowed from type 'ssize_t' (aka 'long') to 'int' in initializer list [-Wc++11-narrowing]. V1_stride0, V1_stride1.             ^~~~~~~~~~. /Users/foh/.theano/compiledir_macOS-10.14.6-x86_64-i386-64bit-i386-3.8.2-64/tmpzsu2aa1s/mod.cpp:520:13: note: insert an explicit cast to silence this issue. V1_stride0, V1_stride1.             ^~~~~~~~~~.             static_cast<int>( ). 5 errors generated.. ", '[Elemwise{mul,no_inplace}(<TensorType(float64, row)>, <TensorType(float64, (True, True))>)]')

exhibit B

model.py", line 47, in <module>
    trace = pm.sample()
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/pymc3/sampling.py", line 469, in sample
    trace = _mp_sample(**sample_args)
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/pymc3/sampling.py", line 1053, in _mp_sample
    sampler = ps.ParallelSampler(
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/pymc3/parallel_sampling.py", line 355, in __init__
    self._samplers = [
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/pymc3/parallel_sampling.py", line 356, in <listcomp>
    ProcessAdapter(
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/site-packages/pymc3/parallel_sampling.py", line 242, in __init__
    self._process.start()
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 42, in _launch
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/multiprocessing/spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "/Users/foh/anaconda3/envs/pymc/lib/python3.8/multiprocessing/spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

Please provide any additional information below.

oh i also updated gcc via homebrew and rebooted macos to no avail

Versions and main components

AlexAndorra commented 4 years ago

Hi Henrik, Thanks for reporting! I'm closing since I think this is fixed by #3919, but feel free to reopen if needed.