Populate local context with variables from imported notebook

[dschien]

Hi,

I would like to see the capability to have variables defined in my imported notebook (e.g. in # cells) to also be available in my importing notebook. This feature would allow me to split up logic from large notebooks into smaller notebooks.

At the moment, I need to inject and execute all code into my importing notebook in order to get access to any variable state.

Any comments if such an option would be desirable/feasible?

Many thanks.

[parente]

Thanks for opening this. I think I'm missing what you're asking, but let me try to answer and you can tell me where I'm off base.

If you annotate a cells with # <api> in notebook A and import them into notebook B like import mywb.notebook_a as a, then those annotated cells in A execute on import into B and any in-memory state is available to A through the module namespace a. The behavior is like a regular Python module on import, but for # <api> annotated cells only.

[dschien]

Thank you for clarifying. The solves my problem. Specifically, I just tried from a import * and that seems to work for introducing all variables defined in a into b. Interestingly, I could not use mywb.a. With that prefix the module could not be found. I run jupyter_cms-0.3.0.

[parente]

Hm. Interesting that the prefix did not work. What platform are you on? There's a defect open about Windows and I wonder if this is a hint about what's wrong there.

[dschien]

I'm on OSX. What sort of information might be insightful?

[parente]

I've got a Mac so let me try that here before I bug you for more info.

[parente]

Hm. I just did:

virtualenv ~/.virtualenv/notebook-stable
source ~/.virtualenv/notebook-stable/bin/activate
pip install jupyter
pip install jupyter_cms

Then I created a.ipynb with content:

# <api>
x = 1

I next created b.ipynb with content:

%load_ext urth.cms

import mywb.a as a

a.x

It works and shows the output as 1. I also tried:

%load_ext urth.cms

from mywb.a import *

x

which works as well.

Finally, I tried:

%load_ext urth.cms

from a import *

x

which also works, but shouldn't by design.

[dschien]

Is jupyter_cms python 3 - only and I have missed that? These steps work fine with python 3.5 but before I did the above with python 2.7.10 and none worked.

with

import mywb.a as a

a.x

I get

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-1-bda984da512f> in <module>()
      3 import mywb.a as a
      4 
----> 5 a.x

AttributeError: 'module' object has no attribute 'x'

In [2]:

with

from mywb.a import *

x

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-2-e978e5265095> in <module>()
      1 from mywb.a import *
      2 
----> 3 x

NameError: name 'x' is not defined

With

from a import *

x

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-3-7b733649dedc> in <module>()
      3 from a import *
      4 
----> 5 x

NameError: name 'x' is not defined

[parente]

The plot thickens: It should work in 2.7 too. In fact, my Mac test above was run against 2.7.

[parente]

Are you annotating the cell with x=1 in a.ipynb properly? Saving it before trying to load it in b? Restarting the kernel in b if you make changes in a because it behaves like a normal python module and is cached in sys.modules?

[dschien]

Oh my, this is getting more and more obscure. I activated 2.7 venv again and now your original code runs fine. So I went back to my actual work and again module not found.

I then copied the imported file (base) from my work folder where a and b are located and imported base from b and that also worked.

Now the weird thing: I copied a, b, and base into a subfolder, started the notebook server from there and now importing a or base no more works with import mywb.base as a or from mywb.a import * (in both cases ImportError: No module named). However, from a import * still works fine!

[parente]

If you start introducing subfolders, you need to make sure you add the subfolder to any absolute module import. So if you've now got:

folder_where_notebook_started/
  some_subfolder/
    a.ipynb
    b.ipynb
    base.ipynb

and you're doing the import in a.ipynb, you need to write it as import mywb.some_subfolder.base. The imports are absolute, not relative.

If the module magic is getting you down, there's also a function form of the same capability which doesn't have any module magic. It's described in the tutorial here https://github.com/jupyter-incubator/contentmanagement/blob/master/etc/notebooks/cookbooks_demo/use_cookbooks.ipynb.

All that said, from a import * should not be working at all. I'll have to look into that.

[dschien]

Sorry, I wasn't clear enough. I started the notebook server in the subfolder. In other words. I moved the files to the subfolder, quite the original server, cd'ed into the subfolder some_subfolder, started the server on the same port.

So, the notebook index shows these files directly - not inside of a folder.

    a.ipynb
    b.ipynb
    base.ipynb

I thought there is some reference to the path in the metadata but a grep did not show anything. I turned on the chrome dev tools, to disable javascript caching. I have no explanation.

Regarding the function form - this is what I started with, but that did not allow me to execute the imported notebook (e.g. a) and populate the importing context (e.g. b) so that the variables from a are available directly . They would be bound to the name of the imported notebook (i.e. x as defined in a becomes available in b via a.x and not as just x), no?

[parente]

I thought there is some reference to the path in the metadata but a grep did not show anything. I turned on the chrome dev tools, to disable javascript caching. I have no explanation.

OK. Looks like I have more digging to do.

Regarding the function form - this is what I started with, but that did not allow me to execute the imported notebook (x from a) and populate the importing context (x available in b) with the variables from a. They would be bound to the name of the imported notebook (i.e. x is available in b via a.x), no?

That's right. You get a module back with all the objects from that notebook hanging off it. You might be able to play tricks with a loop to copy everything from that module into the local globals, but, yuck. :)