Widgets are no longer included with ipython-notebook.

[moorepants]

Before IPython 4.0 when ipython-notebook was installed the widget functionality was included. But now when I install ipython-notebook the widgets are no longer included.

moorepants@caramelmonkey:~$ conda update conda
Fetching package metadata: ......
# All requested packages already installed.
# packages in environment at /home/moorepants/miniconda:
#
conda                     3.18.1                   py27_0  
moorepants@caramelmonkey:~$ conda create -n widgets-test python ipython-notebook
Fetching package metadata: ......
Solving package specifications: ..............
Package plan for installation in environment /home/moorepants/miniconda/envs/widgets-test:

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    jupyter_client-4.1.1       |           py27_0          89 KB
    nbformat-4.0.1             |           py27_0         112 KB
    ------------------------------------------------------------
                                           Total:         201 KB

The following NEW packages will be INSTALLED:

    certifi:            14.05.14-py27_0
    decorator:          4.0.4-py27_0   
    ipykernel:          4.1.0-py27_0   
    ipython:            4.0.0-py27_0   
    ipython-notebook:   4.0.4-py27_0   
    ipython_genutils:   0.1.0-py27_0   
    jinja2:             2.8-py27_0     
    jsonschema:         2.4.0-py27_0   
    jupyter_client:     4.1.1-py27_0   
    jupyter_core:       4.0.6-py27_0   
    libsodium:          0.4.5-0        
    markupsafe:         0.23-py27_0    
    mistune:            0.7.1-py27_0   
    nbconvert:          4.0.0-py27_0   
    nbformat:           4.0.1-py27_0   
    notebook:           4.0.5-py27_0   
    openssl:            1.0.1k-1       
    path.py:            8.1.2-py27_0   
    pexpect:            3.3-py27_0     
    pickleshare:        0.5-py27_0     
    pip:                7.1.2-py27_0   
    ptyprocess:         0.4-py27_0     
    pygments:           2.0.2-py27_0   
    python:             2.7.10-1       
    pyzmq:              14.7.0-py27_0  
    readline:           6.2-2          
    setuptools:         18.3.2-py27_0  
    simplegeneric:      0.8.1-py27_0   
    sqlite:             3.8.4.1-1      
    ssl_match_hostname: 3.4.0.2-py27_0 
    terminado:          0.5-py27_0     
    tk:                 8.5.18-0       
    tornado:            4.2.1-py27_0   
    traitlets:          4.0.0-py27_0   
    wheel:              0.26.0-py27_1  
    zeromq:             4.0.5-0        
    zlib:               1.2.8-0        

Fetching packages ...
jupyter_client 100% |#########################################################################################################################################| Time: 0:00:00 483.14 kB/s
nbformat-4.0.1 100% |#########################################################################################################################################| Time: 0:00:00 442.53 kB/s
Extracting packages ...
[      COMPLETE      ]|############################################################################################################################################################| 100%
Linking packages ...
[      COMPLETE      ]|############################################################################################################################################################| 100%
#
# To activate this environment, use:
# $ source activate widgets-test
#
# To deactivate this environment, use:
# $ source deactivate
#
moorepants@caramelmonkey:~$ act widgets-test 
discarding /home/moorepants/miniconda/bin from PATH
prepending /home/moorepants/miniconda/envs/widgets-test/bin to PATH
(widgets-test)moorepants@caramelmonkey:~$ ipython
Python 2.7.10 |Continuum Analytics, Inc.| (default, Sep 15 2015, 14:50:01) 
Type "copyright", "credits" or "license" for more information.

IPython 4.0.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: from IPython.html import widgets
/home/moorepants/miniconda/envs/widgets-test/lib/python2.7/site-packages/IPython/html.py:14: ShimWarning: The `IPython.html` package has been deprecated. You should import from `notebook` instead. `IPython.html.widgets` has moved to `ipywidgets`.
  "`IPython.html.widgets` has moved to `ipywidgets`.", ShimWarning)

In [2]: dir(widgets)
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-2-15203d59e637> in <module>()
----> 1 dir(widgets)

/home/moorepants/miniconda/envs/widgets-test/lib/python2.7/site-packages/IPython/utils/shimmodule.pyc in __dir__(self)
     73 
     74     def __dir__(self):
---> 75         return dir(__import__(self._mirror))
     76 
     77     @property

ImportError: No module named ipywidgets

In [3]: 

I think that the new ipywidgets package should be a dependency of ipython-notebook.

[msarahan]

I'm not sure this is a good thing. I think it should be an optional add-on (ideally done with something like https://github.com/conda/conda/issues/1696 ), but not a hard requirement. The whole point of the IPython project split-up was to decouple these things.

[moorepants]

Yes, the decouple is fine but when it leads to backwards incompatible breaks it isn't fine. IPython 4 includes a "shim" so that import IPython.html.widgets still works, but only if ipywidgets is installed. The IPython 3 behavior was that import IPython.html.widgets works and if IPython 4 is backwards compatible, then it should work there too. So the IPython 4 dependencies for pip install ipython[notebook] and conda install ipython-notebook should work in a backwards compatible way. Currently the conda install doesn't. I haven't checked the pip install command.

[moorepants]

And yes, some kind of optional dependency feature in conda would be a good thing. pip has this, as noted above and it seems like a nice way to handle things like this.

[msarahan]

pip install ipython notebook

Does not install widgets. Sorry, but this is really upstream breaking backwards compatibility. Please, by all means, complain to the IPython team. I really don't think conda should "fix" this by forcing the widgets install (but optional dependency would be OK.)

[moorepants]

It's not upstream. If anaconda makes their IPython package have a dependency on ipywidgets, it is fixed. I'll send a message to IPython too.

Do you work for continuum or something now?

[msarahan]

The decision to make notebook NOT explicitly depend on widgets was upstream. Continuum is only reflecting that upstream decision.

Yes, I've been with Continuum for about 2 months now.

[moorepants]

Congrats on the new job. I just filed an issue at IPython.

[moorepants]

Sorry to beat a dead horse, but doesn't Continuum decide what dependencies are included for the ipython-notebook package. I'm sure you all have a look at the dependencies that IPython uses with setuptools/distutils but you are ultimately creating a new package for anaconda called ipython-notebook that can be whatever you want. For example, if I conda install matplotlib it installs optional dependencies as hard ones, e.g. qt, whereas if I pip install matplotlib those things are optional. I see the anaconda packages as independent of what is specified in the particular Python package's setup.py file. You all get to choose whatever you want for dependencies to make the life of anaconda users easier. And you seem to do so with other packages. You also even rename packages such that they don't match a pip installable equivalent.

[moorepants]

FYI, the IPython folks are fixing this: https://github.com/ipython/ipython/issues/8905#issuecomment-147798570

[msarahan]

Great, we'll follow the IPython team's lead.

WRT repackaging: yes, we can repackage things however. I think the real question here is the reasoning behind decisions like IPython's split of packages. Splitting widgets off was done for good reason. I don't know their exact reasons, but I would do such a thing to make it easier (conceptually, at the very least) to plug in some other widget framework. Decoupling things is good. If Continuum repackages things in a way different from upstream, then that leads to all kinds of headache of "who installed what", and "why is package X not installed with pip, when it is with conda?" Better to match "official" distributions for these things.

As for renaming and repackaging, my personal opinion is that this confuses things, and I hope we'll avoid that going forward. I will keep an eye out for it.