An error occurred in the 'portfolio_performance.ipynb',

hellojinwoo commented 5 years ago

Issue

While executing the codes in the file 'porfolio_performance.ipynb', I encountered an error message saying <timed exec> in <module>.
Guess the pystan library has some problem because the last excerpt of codes is from the pystan.py
For your information, pystan's version : 2.18.0.0 & pandas' version : 0.23.4 ( I created a virtual environment and download dependent libraries from your 'requirements.txt' )

Background Information

I am running your codes in the following environment:

OS : Ubuntu 18.04.2 LTS (GNU/Linux 4.15.0-47-generic x86_64)
IDE : Jupyter Lab
Python : '3.7.3 (default, Mar 27 2019, 22:11:17)

Error message

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<timed exec> in <module>

~/anaconda3/envs/GPLVM/lib/python3.7/site-packages/ipyparallel/client/view.py in push(self, ns, targets, block, track)
    708         if not isinstance(ns, dict):
    709             raise TypeError("Must be a dict, not %s"%type(ns))
--> 710         return self._really_apply(util._push, kwargs=ns, block=block, track=track, targets=targets)
    711 
    712     def get(self, key_s):

<decorator-gen-139> in _really_apply(self, f, args, kwargs, targets, block, track)

~/anaconda3/envs/GPLVM/lib/python3.7/site-packages/ipyparallel/client/view.py in sync_results(f, self, *args, **kwargs)
     50     self._in_sync_results = True
     51     try:
---> 52         ret = f(self, *args, **kwargs)
     53     finally:
     54         self._in_sync_results = False

<decorator-gen-138> in _really_apply(self, f, args, kwargs, targets, block, track)

~/anaconda3/envs/GPLVM/lib/python3.7/site-packages/ipyparallel/client/view.py in save_ids(f, self, *args, **kwargs)
     35     n_previous = len(self.client.history)
     36     try:
---> 37         ret = f(self, *args, **kwargs)
     38     finally:
     39         nmsgs = len(self.client.history) - n_previous

~/anaconda3/envs/GPLVM/lib/python3.7/site-packages/ipyparallel/client/view.py in _really_apply(self, f, args, kwargs, targets, block, track)
    558         pf = PrePickled(f)
    559         pargs = [PrePickled(arg) for arg in args]
--> 560         pkwargs = {k: PrePickled(v) for k, v in kwargs.items()}
    561 
    562         for ident in _idents:

~/anaconda3/envs/GPLVM/lib/python3.7/site-packages/ipyparallel/client/view.py in <dictcomp>(.0)
    558         pf = PrePickled(f)
    559         pargs = [PrePickled(arg) for arg in args]
--> 560         pkwargs = {k: PrePickled(v) for k, v in kwargs.items()}
    561 
    562         for ident in _idents:

~/anaconda3/envs/GPLVM/lib/python3.7/site-packages/ipyparallel/serialize/serialize.py in __init__(self, obj)
     42     """
     43     def __init__(self, obj):
---> 44         self.buffers = serialize_object(obj)
     45 
     46 

~/anaconda3/envs/GPLVM/lib/python3.7/site-packages/ipyparallel/serialize/serialize.py in serialize_object(obj, buffer_threshold, item_threshold)
    123         buffers.extend(_extract_buffers(cobj, buffer_threshold))
    124 
--> 125     buffers.insert(0, pickle.dumps(cobj, PICKLE_PROTOCOL))
    126     return buffers
    127 

~/anaconda3/envs/GPLVM/lib/python3.7/site-packages/pystan/model.py in __getstate__(self)
    394         """
    395         state = self.__dict__.copy()
--> 396         del state['module']
    397         del state['fit_class']
    398         return state

KeyError: 'module'

Added Notice

I just found that this error occurs every .ipynb file (model_evaluation, latent_space_embedding...). How can I fix this problem?

rsnirwan commented 5 years ago

Did you activate the virtualenv before you start the cluster? Otherwise the clusters might work with a different version of the packages than main notebook. I just did a test run on Ubuntu and MacOS with python 3.7.2 in a virtualenv (with packages from requirements.txt) and every notebook is working fine.

Here are the steps I did: git clone https://github.com/RSNirwan/GPLVMsInFinance/ cd GPLVMsInFinance/ virtualenv -p python3 . . bin/activate pip install -r requirements.txt jupyter notebook

In another terminal: cd GPLVMsInFinance/ . bin/activate ipcluster start -n 4

hellojinwoo commented 5 years ago

Did you activate the virtualenv before you start the cluster? Otherwise the clusters might work with a different version of the packages than main notebook.

This tip helped me solve the 'module' problem.

I reinstalled the anaconda on my Ubuntu and as you advised, activated the virtualenv before starting clusters on the different cmd.

Thank you!

rsnirwan / GPLVMsInFinance