Use of plotly-notebook-js to draw charts (check your AdBlock/Privacy Badger!)

eugenesvk commented 8 years ago

I was trying to use jupyter notebook with a plotly JS addon plotly-notebook-js to be able to draw plotly charts locally using IJavascript engine.

However, I realized that it requires this addon to be installed locally in the path I launch jupyter notebook from (see this issue for more details).

As far as I understood from notebook issue IJavascript is responsible for this. So I'd like to request a feature that would read global node modules while running a notebook with IJavascript kernel so that I don't have to do npm install plotly-notebook-js in every place I have a notebook in. Thank you!

n-riesco commented 8 years ago

cc/ @mdtusz

IJavascript can load local and global modules.

The issue lies in the method to inject Plotly.js into the notebook.

Here's a workaround that takes advantage of some of the new features available in the beta version of IJavascript:

$$.html('<script>require(["https://cdn.plot.ly/plotly-latest.min.js"], function(Plotly) { if (Plotly) window.Plotly = Plotly; })</script>');

var Plot = require('plotly-notebook-js');
var NotebookPlot = Plot.createPlot().constructor;
NotebookPlot.prototype._toHtml = NotebookPlot.prototype.render;

Plot.createPlot([{ x: [1,2,3], y: [3,4,5] }], { title: 'Plotly in Jupyter!' });

untitled

eugenesvk commented 8 years ago

Unfortunately, this doesn't work — I still need plotly-notebook-js as a local module. I get the following error when trying to run your script (I have ijavascript@5.0.11-beta.0 installed) 404 GET /notebooks/node_modules/plotly-notebook-js/build/plotly.min.js?v=20160330154650 (::1): No such file or directory: node_modules/plotly-notebook-js/build/plotly.min.js

By the way, is it possible to output to an svg/png file from within the notebook? Or is it something only plotly-notebook-js knows about?

n-riesco commented 8 years ago

That error message doesn't look like one of the tracebacks that IJavascript generates.

Please, could you describe what steps you're following to run the code?

eugenesvk commented 8 years ago

Certainly.

CD into the folder where I have my notebook file in PlotlyNotebook.ipynb
Run jupyter notebook
A new http://localhost:8888/tree opens in Chrome
I click PlotlyNotebook.ipynb in the list of notebooks
It opens in a new tab http://localhost:8888/notebooks/PlotlyNotebook.ipynb
I see empty output Out [1] (duplicate) right away and also when I hit Shift+Enter

If I do the same things, but have Notebook\node_modules\plotly-notebook-js\ then everything is fine.

n-riesco commented 8 years ago

You need to install plotly-notebook-js either locally, like this:

$ cd && npm install plotly-notebook-js

or globally:

$ sudo npm install -g plotly-notebook-js

Once plotly-notebook-js is installed, you can test whether it's accessible from Node.js by running the following:

$ node
> require("plotly-notebook-js")
{ createPlot: [Function: NotebookPlot] }

If Node doesn't see the package, IJavascript won't either.

eugenesvk commented 8 years ago

Node can see the global package, but IJavascript can't This is my command prompt without local plotly-notebook-js module

@PC C:\..\_TestCharts\Notebook> npm list
C:\Dev\_TestCharts\Notebook
└── (empty)

@PC C:\..\_TestCharts\Notebook> node     
> require("plotly-notebook-js")          
{ createPlot: [Function: NotebookPlot] } 
>

Just FYI it's exactly the same with local module

@PC C:\..\_TestCharts\Notebook> npm list 
C:\Dev\_TestCharts\Notebook              
└── plotly-notebook-js@0.1.2             

@PC C:\..\_TestCharts\Notebook> node     
> require("plotly-notebook-js")          
{ createPlot: [Function: NotebookPlot] } 
> .exit

n-riesco commented 8 years ago

This is very intriguing!

What output do you get when you run require("plotly-notebook-js") in the notebook?

Could you describe your installation? Is this machine Windows? How did you install Node? How did you install IJavascript?

eugenesvk commented 8 years ago

Was just adding a P.S. "To avoid confusion, this is on Windows 10 (and apologies for not mentioning it earlier) as I see you're suggesting *nix commands with sudo." when I saw your comment :) I'll check this on a OSX I have access to to make sure this is a Windows issue.

By the way, I also get { createPlot: [Function: NotebookPlot] } when I type just require('plotly-notebook-js'); in the notebook, so it seems that I was wrong to suggest that IJavascript doesn't see global modules. I still have no clue what the issue is :(, hopefully we can hunt it down and fix it with your kind help

Meanwhile my Windows setup is as follows: Microsoft Windows [Version 10.0.10586] Node was installed via nodist, with the latest version 5.9.1 IJavascript was installed with a simple npm install -g ijavascript and resides in the default global module location c:\Dev\Node.js\bin\node_modules\ijavascript\ Jupyter was installed via default conda install jupyter in the default conda3 (i.e. python3) environment. Let me know if you need anything more specific for this.

eugenesvk commented 8 years ago

Maybe this is actually a plotly-notebook-js issue despite the suggestion to the contrary by @mdtusz in his issue comment. Is there a simple way to test this so I don't have to waste your time and reopen my original issue?

n-riesco commented 8 years ago

@eugenesvk I'm interested in making plotly-notebook-js work in IJavascript.

I'm going to sum up the situation. Please, correct me if I'm wrong:

Windows 10, nodist and conda3
npm install -g ijavascript
when you run the example I posted in this comment you get two empty Out[1]:. That is, IJavascript doesn't show any result or error.

Assuming, you are running the notebook in Firefox or Chrome. Right after running the code in the notebook, please, open the dev console by pressing shift-ctrl-i and click on the Console tab:

can you see any error messages?
if you type Plotly in the dev console, what do you get?

eugenesvk commented 8 years ago

On OSX I have an even stranger issue. jupyter notebook can't see IJavascript kernel by itself (using nodenv as a version manager here, but otherwise the install steps are the same), it throws kernel matching Javascript (Node.js). When I run ijs to launch it with IJavascript kernel by default I get a dead kernel The kernel appears to have died. It will restart automatically, though it dies again once I refresh and click on the field with the code (or just wait 2 seconds). I'll stop with OSX for now and focus on Widows to try to resolve one issue at a time.

Here is my terminal output

Repos/Notebook > ijs                                                                                               3m44s
[TerminalIPythonApp] WARNING | Subcommand `ipython notebook` is deprecated and will be removed in future versions.
[TerminalIPythonApp] WARNING | You likely want to use `jupyter notebook`... continue in 5 sec. Press Ctrl-C to quit now.
[I 17:43:45.868 NotebookApp] Serving notebooks from local directory: /Users/eugenesv/Documents/Dev/Repos/Notebook
[I 17:43:45.869 NotebookApp] 0 active kernels
[I 17:43:45.869 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/
[I 17:43:45.869 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[W 17:44:05.313 NotebookApp] Notebook PlotlyNotebook-Test.ipynb is not trusted
[I 17:44:06.321 NotebookApp] Kernel started: d069d3ca-495b-4a25-a030-af4cc6c19b06
[I 17:44:09.312 NotebookApp] KernelRestarter: restarting kernel (1/5)
WARNING:root:kernel d069d3ca-495b-4a25-a030-af4cc6c19b06 restarted

n-riesco commented 8 years ago

One more thing. If the notebook failed to load Plotly, you may need to close the browser and open it again. It is important that this line:

$$.html('<script>require(["https://cdn.plot.ly/plotly-latest.min.js"], function(Plotly) { if (Plotly) window.Plotly = Plotly; })</script>');

is run before calling Plotly.

eugenesvk commented 8 years ago

You're correct in describing my setup (conda is version 4.0.5, but it's conda3 in a sense that the default environment is python3). I'm using Chrome 49.0.2623.110 m (64-bit). And yes, I get two empty output lines without any errors. Your example is copy&pasted without any modifications.

My console output before typing plotly is as follows

Failed to load resource: net::ERR_BLOCKED_BY_CLIENT
Default extension for cell metadata editing loaded.
Raw Cell Format toolbar preset loaded.
Slideshow extension for metadata editing loaded.
actions jupyter-notebook:find-and-replace does not exist, still binding it in case it will be defined later...
Session: kernel_created (445d99bb-d967-4847-bfaa-82bf924303f3)
Starting WebSockets: ws://localhost:8888/api/kernels/cab83214-f5b1-4d71-8dd9-7d6b9b99fc46
Failed to load resource: net::ERR_BLOCKED_BY_CLIENT
Uncaught Error: Script error for "https://cdn.plot.ly/plotly-latest.min.js"
http://requirejs.org/docs/errors.html#scripterror
Failed to load resource: the server responded with a status of 404 (Not Found)
Uncaught Error: Script error for "/notebooks/node_modules/plotly-notebook-js/build/plotly.min.js"
http://requirejs.org/docs/errors.html#scripterror
Loading extension: widgets/notebook/js/extension
loaded widgets
Kernel: kernel_connected (cab83214-f5b1-4d71-8dd9-7d6b9b99fc46)
Kernel: kernel_ready (cab83214-f5b1-4d71-8dd9-7d6b9b99fc46)
0
Failed to load resource: net::ERR_BLOCKED_BY_CLIENT
Failed to load resource: net::ERR_BLOCKED_BY_CLIENT
Failed to load resource: the server responded with a status of 404 (Not Found)
0

When I type plotly I get the following

plotly
Uncaught ReferenceError: plotly is not defined(…) VM368:2 
(anonymous function) @ VM368:2
InjectedScript._evaluateOn @ VM333:878
InjectedScript._evaluateAndWrap @ VM333:811
InjectedScript.evaluate @ VM333:667

Though it's better to check out screenshots as they're colored and easier to read (3 screenshots so there might be some overlap in top/bottom lines) plotlyconsole1 plotlyconsole2 plotlyconsole3

n-riesco commented 8 years ago

@eugenesvk The OS X issue is known ("cheeky" Jupyter developers and their 5-second prodding!), I will fix it in the coming release.

eugenesvk commented 8 years ago

Hm, when I clear console and try to re-run the notebook code again, I get no errors in the console, just an incremental number (1,2,3) depending on how many times I've hit shift-Enter Relaunched again and this errors do appear when I open the notebook, so they might still be relevant

n-riesco commented 8 years ago

I guess the error message Failed to load resource: net::ERR_BLOCKED_BY_CLIENT means something is blocking the request to download Plotly. I also see in the screenshot that a connection to kaspersky-labs.com is also blocked. Is this perhaps an issue with an antivirus?

n-riesco commented 8 years ago

Adblock Plus?

eugenesvk commented 8 years ago

Amg, just found out by following a link to the addon that appeared in console - it was stupid privacy badger blocking it. I still don't get why it wasn't blocking it when I had the module installed locally, but doesn't matter at this point. Will just disable it.

n-riesco commented 8 years ago

See here.

eugenesvk commented 8 years ago

Thanks for the extra info! I wouldn't think of that because it worked with the local module By the way, all my previous examples (where I didn't have to use $$.html('<script>require(["https://cdn.plot.ly/plotly-latest.min.js"], function(Plotly) { if (Plotly) window.Plotly = Plotly; })</script>'); and would just do a regular output at the end $$html$$ = myPlot.render();) now work fine. Is there any reason to prefer one way or another?

And the last thing, maybe you could help with the only issue remaining for me — is it possible to output to an svg/png file from within the notebook? Tried Plotly.Snapshot.toSVG(gd); or the media function from one of the examples, but they don't seem to work (maybe just not defined in the plotly-notebook-js wrapper or I'm just doing something wrong.

n-riesco commented 8 years ago

@eugenesvk I'm deprecating the use of global objects like $$html$$ because I want to add more without polluting the global scope. The plan now is to offer the same functionality through a single global $$.

And to answer your last question: I reckon it's possible. I will have a look a plotly-notebook-js in the future, but I have to complete other pressing issues first (like the nagging 5s delay from ipython notebook :tired_face: ).

mdtusz commented 8 years ago

@eugenesvk the Plot object that is used in plotly-notebook-js doesn't expose any export functionality, but open an issue there with more details of what you're hoping for and I'll be happy to look into it when I've got a moment.

eugenesvk commented 8 years ago

@n-riesco ok, thanks for clarification, will change my notebooks to your new method. Totally understand the priorities, just keep that in mind when you squash other bugs :) Meanwhile will open an issue with @mdtusz who kindly offered to look into it.

eugenesvk commented 8 years ago

Hm, this doesn't seem to be resolved :( I've changed the first line to require plotly.js locally instead of from CDN so I can actually work offline $$.html('<script>require(["plotly.js"], function(Plotly) { if (Plotly) window.Plotly = Plotly; })</script>'); and it only works when I have plotly-notebook-js installed as a local module, not when it's only available globally. Even though separate require command (inside Notebook) gives out the same output in both cases.

require("plotly-notebook-js")          
{ createPlot: [Function: NotebookPlot] }

Have no idea why this is happening :( Please note the difference in error logs that does seem to suggest that plotly-notebook-js can't be properly loaded unless it's installed locally (and it's being searched for in a local path /notebooks/node_modules/plotly-notebook-js. But then again requiring it gives the { createPlot: [Function: NotebookPlot] } output

Here are errors when I run it withplotly-notebook-js installed only globally

-Screenshot of Chrome console plotlyconsole0_global

Command prompt log while running the notebook

PC C:\..\_TestCharts\Notebook (@1.0.0)> jupyter notebook
[I 03:34:45.636 NotebookApp] Serving notebooks from local directory: C:\Dev\_TestCharts\Notebook
[I 03:34:45.639 NotebookApp] 0 active kernels
[I 03:34:45.640 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/
[I 03:34:45.641 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 03:34:50.125 NotebookApp] Kernel started: 341f9111-064d-4018-84f1-0ecbd7726b72
[W 03:34:50.156 NotebookApp] 404 GET /notebooks/plotly.js?v=20160331033445 (::1): No such file or directory: plotly.js
[W 03:34:50.175 NotebookApp] 404 GET /notebooks/plotly.js?v=20160331033445 (::1) 20.02ms referer=http://localhost:8888/notebooks/PlotlyNotebook-Test.ipynb
[W 03:34:50.183 NotebookApp] 404 GET /notebooks/node_modules/plotly-notebook-js/build/plotly.min.js?v=20160331033445 (::1): No such file or directory: node_modules/plotly-notebook-js/build/plotly.min.js
[W 03:34:50.193 NotebookApp] 404 GET /notebooks/node_modules/plotly-notebook-js/build/plotly.min.js?v=20160331033445 (::1) 11.01ms referer=http://localhost:8888/notebooks/PlotlyNotebook-Test.ipynb
KERNEL: SHELL_SOCKET: Unhandled message type: comm_open
[W 03:34:53.321 NotebookApp] 404 GET /static/components/backbone/backbone-min.map (::1) 2.00ms referer=None
[W 03:34:53.914 NotebookApp] 404 GET /notebooks/plotly.js?v=20160331033445 (::1): No such file or directory: plotly.js
[W 03:34:53.919 NotebookApp] 404 GET /notebooks/plotly.js?v=20160331033445 (::1) 6.01ms referer=http://localhost:8888/notebooks/PlotlyNotebook-Test.ipynb
[W 03:34:53.925 NotebookApp] 404 GET /notebooks/node_modules/plotly-notebook-js/build/plotly.min.js?v=20160331033445 (::1): No such file or directory: node_modules/plotly-notebook-js/build/plotly.min.js
[W 03:34:53.934 NotebookApp] 404 GET /notebooks/node_modules/plotly-notebook-js/build/plotly.min.js?v=20160331033445 (::1) 10.01ms referer=http://localhost:8888/notebooks/PlotlyNotebook-Test.ipynb
[I 03:35:55.406 NotebookApp] Kernel shutdown: 341f9111-064d-4018-84f1-0ecbd7726b72
[I 03:36:00.645 NotebookApp] Interrupted...
[I 03:36:00.646 NotebookApp] Shutting down kernels

And the same thing when I do have plotly-notebook-js installed as a local module

Screenshot of Chrome console
Command prompt log

PC C:\..\_TestCharts\Notebook (@1.0.0)> jupyter notebook
[I 03:32:33.559 NotebookApp] Serving notebooks from local directory: C:\Dev\_TestCharts\Notebook
[I 03:32:33.561 NotebookApp] 0 active kernels
[I 03:32:33.562 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/
[I 03:32:33.563 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 03:32:37.719 NotebookApp] Kernel started: d3f80ea6-e997-4cf5-b9e2-5909b672b6c1
[W 03:32:37.766 NotebookApp] 404 GET /notebooks/plotly.js?v=20160331033233 (::1): No such file or directory: plotly.js
[W 03:32:37.781 NotebookApp] 404 GET /notebooks/plotly.js?v=20160331033233 (::1) 15.35ms referer=http://localhost:8888/notebooks/PlotlyNotebook-Test.ipynb
[I 03:32:37.789 NotebookApp] 302 GET /notebooks/node_modules/plotly-notebook-js/build/plotly.min.js?v=20160331033233 (::1) 2.01ms
KERNEL: SHELL_SOCKET: Unhandled message type: comm_open
[W 03:32:41.520 NotebookApp] 404 GET /notebooks/plotly.js?v=20160331033233 (::1): No such file or directory: plotly.js
[W 03:32:41.526 NotebookApp] 404 GET /notebooks/plotly.js?v=20160331033233 (::1) 7.02ms referer=http://localhost:8888/notebooks/PlotlyNotebook-Test.ipynb
[I 03:32:41.535 NotebookApp] 302 GET /notebooks/node_modules/plotly-notebook-js/build/plotly.min.js?v=20160331033233 (::1) 2.51ms
[W 03:32:41.706 NotebookApp] 404 GET /static/components/backbone/backbone-min.map (::1) 2.00ms referer=None
[I 03:33:20.676 NotebookApp] Kernel shutdown: d3f80ea6-e997-4cf5-b9e2-5909b672b6c1
[I 03:33:28.567 NotebookApp] Interrupted...
[I 03:33:28.568 NotebookApp] Shutting down kernels

n-riesco commented 8 years ago

I would expect $$.html('<script>require(["plotly.js"], function(Plotly) { if (Plotly) window.Plotly = Plotly; })</script>'); to fail regardless of whether plotly-notebook-js is installed locally or globally.

I'll try to explain briefly why:

the line $$.html("<script>require('plotly.js')</script>"); injects javascript that is run by the jupyter notebook
the line var Plot = require('plotly-notebook-js');, however, is run by IJavascript This difference is crucial, because it means require('plotly-notebook-js') will use Node.js to locate the package 'plotly-notebook-js' in your computer, whereas require('plotly.js') doesn't.

Basically, what you're trying to achieve now is that the Jupyter notebook comes with plotly.js preloaded. I think this is possible with a notebook extension. I'm not sure how up to date the previous link but I hope this gives enough info to get you started.

eugenesvk commented 8 years ago

Ok, thanks, that's too complicated for me, will wait for a better integration with the notebook that would allow full plotly.js power inside and meanwhile just use nw.js to test/export images.

n-riesco commented 8 years ago

I'm interested in getting this working. I'll keep this issue open as a reminder.

rgbkrk commented 8 years ago

Just a follow up here - we have a new way of handling plotly outputs in nteract which is with an application/vnd.plotly.v1+json mimetype. Example with Python:

n-riesco commented 8 years ago

~~@rgbkrk Is this implemented via transfomime? If not, could you point me to the code implementing this?~~

https://github.com/jupyter/nbformat/pull/43

n-riesco commented 8 years ago

Not tested, but the following should work with nteract and IJavascript:

function plot(data, layout) {
    $$.mime({
        "application/vnd.plotly.v1+json": {
            data: data,
            layout: layout
        }
    });
}

rgbkrk commented 8 years ago

We could add it to transformime - right now it's a custom transform packaged as part of nteract/nteract.

n-riesco commented 7 years ago

Closed via https://github.com/n-riesco/ijavascript-plotly .

n-riesco / ijavascript

Use of plotly-notebook-js to draw charts (check your AdBlock/Privacy Badger!) #66