Unable to invoke JS9 classes directly within a Jupyter cell's output

[o-smirnov]

So we've got JS9 going if we invoke it within an iframe:

%%html
<iframe src="http://localhost:8888/files/js9.html" width=1000 height=750></iframe>

Direct invocation fails due to a socket.io/XHR issue. See https://github.com/gijzelaerr/js9notebook/issues/2#issuecomment-402548545

@ericmandel suggest we continue this particular discussion here.

[o-smirnov]

OK, I've dreamt up half a solution to this (and #3 #4 #7). Jupyter also serves up files from a /static/ directory, which contains all the extensive HTML/SVG/JS machinery needed to make the notebooks go. This maps to .../lib/python2.7/site-packages/notebook/static, and is obviously not sandboxed.

If I make a symlink from this directory to the JS9 web install directory, I can access the JS9 components via /static/js9-www and voila, WASM works again, and I don't see any more CORS or socket.io errors.

I think this solution is almost totally legit, since we're supposed to be treating JS9 as a trusted internal notebook component anyway. (And in my use case, I've got Jupyter installed in each user's private virtualenv, so the symlink can be created without needing root access.)

So, great news, right? This works perfectly, serving up JS9 via Jupyter:

%%html
<iframe src="/static/js9-www/js9.html" width=800 height=750></iframe>

This gives me a fully-functional JS9 window, I can load files, upload them to the server, etc. Buuuuut.... I need to generate HTML dynamically in my notebooks in order to load the right images, etc. The only way I know of to render dynamically-generated HTML in an iframe is to write it to a file first, and give that file as the src attribute of the iframe tag (but maybe in my ignorance I'm missing another way...) If I generate that file in the notebook working directory, it can only be served up via /files/, and we're back in the sandbox. I suppose I could generate the files inside the notebook/static directory, but that's definitely very kludgy, fragile, and non-legit...

The alternative is to try to escape the iframe, and generate HTML that directly invokes JS9. This almost works via /static/js9-www, but there's still remaining problems.

Using the allinone approach doesn't work at all (cell output is empty):

%%html
<div class="JS9Menubar"></div>
<div class="JS9"></div>
<div style="margin-top: 2px;"><div class="JS9Colorbar"></div></div>
<link type="text/css" rel="stylesheet" href="/static/js9-www/js9-allinone.css">
<script type="text/javascript" src="/static/js9-www/js9-allinone.js"></script>

This code does give me a JS9 window:

%%html
%%html
  <link type="image/x-icon" rel="shortcut icon" href="/static/js9-www/./favicon.ico">
  <link type="text/css" rel="stylesheet" href="/static/js9-www/js9support.css">
  <link type="text/css" rel="stylesheet" href="/static/js9-www/js9.css">
  <link rel="apple-touch-icon" href="/static/js9-www/images/js9-apple-touch-icon.png">
  <script type="text/javascript" src="/static/js9-www/js9prefs.js"></script>
  <script type="text/javascript" src="/static/js9-www/js9support.min.js"></script>
  <script type="text/javascript" src="/static/js9-www/js9.min.js"></script>
  <script type="text/javascript" src="/static/js9-www/js9plugins.js"></script>
  <div class="JS9Menubar"></div>
  <div class="JS9"></div>
  <div style="margin-top: 2px;"><div class="JS9Colorbar"></div></div>
  <script type="text/javascript">
      setTimeout(function loadIm(){
                   JS9.Load("/files/vla_L_xx_re.fits")
                 }, 1000);
  </script>    

But the image does not load, I get a "no FITS module available" dialog and these errors on the JS console:

VM225:3058 Uncaught TypeError: Cannot read property 'mouse' of undefined
    at eval (eval at globalEval (jquery.min.js:2), <anonymous>:3058:25)
    at eval (eval at globalEval (jquery.min.js:2), <anonymous>:3217:3)
    at eval (<anonymous>)
    at Function.globalEval (jquery.min.js:2)
    at text script (jquery.min.js:4)
    at Ab (jquery.min.js:4)
    at z (jquery.min.js:4)
    at XMLHttpRequest.<anonymous> (jquery.min.js:4)
    at XMLHttpRequest.send (<anonymous>:1:781)
    at Object.send (jquery.min.js:4)
VM226:414 Uncaught Error: no FITS module available to process FITS file
    at Object.b.error (eval at globalEval (jquery.min.js:2), <anonymous>:414:470)
    at Object.b.handleFITSFile (eval at globalEval (jquery.min.js:2), <anonymous>:407:362)
    at Object.eval [as Load] (eval at globalEval (jquery.min.js:2), <anonymous>:585:340)
    at XMLHttpRequest.g.onload (eval at globalEval (jquery.min.js:2), <anonymous>:404:80)

Also, with or without that JS9.Load() attempt, the JS9 window is not quite functional. Using the menu results in a whole slew of errors in the JS console:

...and when I try to open a local file, I get back the same "no FITS module available" error as before.

(Except, strangely, this one time the whole browser tab crashed, Chrome asked me to reload, and when I did, JS9 came up with my FITS image, all working and functional.... but it didn't last.... as soon as I refreshed the cell, JS9 went back to its broken state.)

[ericmandel]

This looks very promising and if it does not work out, you can take comfort in having been initialized into a great American theme from my childhood:

https://www.youtube.com/watch?v=055wFyO6gag

I can't find the old reference, but I seem to recall that one early problem was that Jupyter did not always respect that order in which things should be loaded. This would definitely lead to problems.

You can try this for better debugging:

• replace js9.min.js with js9.js, so we can get real line numbers
• then add console.statements to the 'ready' conditions at the bottom of the js9.js file

That will tell us something about the order. The fact that you have no FITS module is very bad and indicates that the Emscripten file (astroem.js without wasm, or astroemw.js with wasm) not get loaded. You can verify which was loaded by inspecting this:

and also check for the window.Module module:

Finally note that there are hacks in js9.js to avoid Jupyter probems. For example, as of yesterday, we turn off wasm support (hence JS9 should load astroem.js instead of astroemw.js).

[o-smirnov]

Yeah there's been several times this week that I've felt like Charlie Brown at the end of that video... :) I think I need to rename this thing to Project Lucy!

The fact that you have no FITS module is very bad

I think this is last night's breakage... will post in the other thread momentarily.

[o-smirnov]

https://www.youtube.com/watch?v=055wFyO6gag

BTW are you deliberately surrounding the links with double ticks? Because if you just paste in the link as is,without any formatting, it becomes clickable, like so: https://www.youtube.com/watch?v=055wFyO6gag

Good news then, the Preload() thing you mentioned in #5 was the key to this problem. This code now executes perfectly and gives me a fully-functional JS9 without iframes, and without any of the problems mentioned above! Fully served by Jupyter, no less.

%%html
  <link type="image/x-icon" rel="shortcut icon" href="/static/js9-www/./favicon.ico">
  <link type="text/css" rel="stylesheet" href="/static/js9-www/js9support.css">
  <link type="text/css" rel="stylesheet" href="/static/js9-www/js9.css">
  <link rel="apple-touch-icon" href="/static/js9-www/images/js9-apple-touch-icon.png">
  <script type="text/javascript" src="/static/js9-www/js9prefs.js"></script>
  <script type="text/javascript" src="/static/js9-www/js9support.min.js"></script>
  <script type="text/javascript" src="/static/js9-www/js9.min.js"></script>
  <script type="text/javascript" src="/static/js9-www/js9plugins.js"></script>
  <div class="JS9Menubar"></div>
  <div class="JS9"></div>
  <div style="margin-top: 2px;"><div class="JS9Colorbar"></div></div>
  <script type="text/javascript">
    JS9.Preload("/files/vla_L_xx_re.fits")
  </script> 

The bad news is, I can only do this once. Re-executing the cells gives me a broken JS9. This is where my HTML-fu gets very uncertain, but I would guess this has something to do with redefining DIV elements with the same names, etc. I can see @mgckind goes to pains to assign unique identifiers in https://github.com/mgckind/jjs9/blob/master/jjs9.py...

[ericmandel]

Thanks for the html tip, if I once knew that markup feature, I've forgotten it. Obviously I do remember the backticks!

What is the expected behavior of re-executing HTML in Jupyter? It's probably doing exactly what its asked to do, but not what you want it to do. for example, element ids should be unique, so re-executing immediately beaks the DOM paradigm. Which may be the point of @mgckind's code.

But I also don't know what happens if you reload the JS9 files again ... do I need to make it skip loading a second time???

[o-smirnov]

What is the expected behavior of re-executing HTML in Jupyter? It's probably doing exactly what its asked to do, but not what you want it to do.

Well it clears the visible browser elements previously created by that bit of HTML, and renders the HTML again. But I'm on thin ice here... don't know much JS and even less about DOM... not sure what happens to existing elements at all!

But I'm pretty sure all your JS9 scripts are reloaded when I do this. Can I add HTML code to the cell that would skip loading a second time? Can you show me how?

[ericmandel]

You can check for the existence of the JS9 object and only load scripts if the object is not present. This is a programmatic way of loading scripts, something akin to what is done here:

https://stackoverflow.com/questions/15521343/conditionally-load-javascript-file https://gomakethings.com/conditionally-loading-javascript-only-when-the-browser-supports-it/

And since you are doing things programmatically, you can also keep a counter of iterations and change the id of the elements. Probably getting close to what @mgckind did ...

[o-smirnov]

Probably getting close to what @mgckind did ...

Yeah I have a feeling I'll be better off adapting his code now rather than reinventing the wheel. Now I have to think a little bit before rushing into this. But this has been a most productive week already!

[ericmandel]

I agree its been a really good week ... and that I also have to do some other stuff right now ...

[o-smirnov]

And on that happy note, I'm going to sign out for the week. Many thanks for all the help @ericmandel, this project is really coming together nicely now... enjoy your weekend!

[ericmandel]

It's great to have someone take up the JS9/Jupyter connection again, so this has been an intense but happy week.

Have a great weekend ... (I originally thought your message said you were out next week)

[ericmandel]

Food for (perhaps near future) thought: why use a JavaScript-based JS9 helper at all? Why not add another helper type to JS9 to communicate directly with Jupyter? JS9 already handles node.js and CGU-based helpers: basically, I only need to implement small number of routines: connect(), send(), blah blah. If, for example, Jupyter is able to receive and reply to standard postMessage messages, I could implement a postMessage helper type, and you could implement (a subset of?) helper requests directly in Python, including the request for image sections. This tighter integration between Jupyter and JS9 might make your life very easy indeed.

One place where this might cause a problem is if you want programs external to Jupyter and JS9 to communicate with JS9 (e.g. command line or pyjs9 communication). I'd have to think a bit, but if there was a way for external Python programs to communicate with Jupyter, we could build on that and have Python, Jupyter, and JS9 all talking to one another.

[o-smirnov]

That's a great idea! There's already a standard way for JS widgets on the client to communicate with the Jupyter's python session on the server (see e.g. http://jupyter.org/widgets), we just have to look into the details. Then indeed on the server side, Jupyter has full access to the images with astropy, and can do everything the helper does -- and will probably be far easier to extend going forward.

This would actually move the project closer to @gijzelaerr's original intention -- a self-contained JS9 Jupyter widget. I think for now I'll develop it within my specific use case to iron out all the kinks, then think about making it more general.

[ericmandel]

Agreed that we ought to get the current paradigm working and then consider how to make a Python helper. But I love the idea of the JS9 helper having access to the full Python/astropy environment.

BTW, I see js2py exists https://github.com/PiotrDabkowski/Js2Py, and I know socket.io for Python exits, which makes me wonder if one can simply run the (converted) helper in Jupyter directly. But probably we want to follow whatever canonical rules are set down by Jupyter.

[o-smirnov]

So I've been trying to adapt @mgckind's paradigm, with unique ID's for the JS9 elements... but getting stuck on another mysterious error message.

I use get_ipython().run_cell_magic('javascript', '', js_command) to inject JS into a cell's output. First I inject this, to create a set of JS9 elements in the output:

    element.append("
            <div id='e25d3a4330b54d81bcd079cb55f7ce3a'>
                <div class='JS9Menubar' id='JS9Menubare25d3a4330b54d81bcd079cb55f7ce3a'></div>
                <div class='JS9' id='JS9e25d3a4330b54d81bcd079cb55f7ce3a'></div>
                <div style='margin-top: 2px;'><div class='JS9Colorbar' id='JS9Colorbare25d3a4330b54d81bcd079cb55f7ce3a'></div></div>
            </div>
            ");

...which results in a greyed-out JS9 window and colorbar. So at least the JS9 elements are created. The next statement attempts to load the image, by injecting this:

JS9.AddDivs('JS9e25d3a4330b54d81bcd079cb55f7ce3a');
JS9.Preload('/files/./3C147-CD-LO-spw0-s7-lwimager.fullrest.fits', {fits2fits:false}, {display: 'JS9e25d3a4330b54d81bcd079cb55f7ce3a'});

and I get this from Jupyter:

Javascript error adding output!
Error: can't find JS9 display with id: JS9Colorbare25d3a4330b54d81bcd079cb55f7ce3a

The error message clearly originates inside JS9, but why is it looking for display ID "JSColorbarXXX" when the Preload call says "JS9XXX"?

[ericmandel]

Try removing the Colorbar div and see if that works, while I re-familiarize myself with code I wrote a few years ago ...

[o-smirnov]

Ah but wait. The error message goes away if the id of the JSColorbar and the JSMenubar is the same as that of the JS9 element. Like so:

element.append("
            <div id='2c95007eed6840029f5c22064b2e02f0'>
                <div class='JS9Menubar' id='JS92c95007eed6840029f5c22064b2e02f0'></div>
                <div class='JS9' id='JS92c95007eed6840029f5c22064b2e02f0'></div>
                <div style='margin-top: 2px;'><div class='JS9Colorbar' id='JS92c95007eed6840029f5c22064b2e02f0'></div></div>
            </div>
            ");

Except now the menu is stuck in the middle of the JS9 window, like so:

And doesn't function. Nor did Preload() seem to have done anything.

[ericmandel]

No, that's not right, here is the correct syntax, which you can see in action in demos like https://js9.si.edu/js9/js9sizes.html:

   <div id="size256">
    <div class="JS9Menubar" id="JS9-256_Menubar" data-buttonClass="JS9Button-flat" data-backgroundColor="lightblue" data-width="256px" data-height="64px"></div>
    <div class="JS9" id="JS9-256" data-width="256px" data-height="256px"></div>
    <div class="JS9Colorbar" id="JS9-256_Colorbar" data-width="256px"></div>
    </div>

[o-smirnov]

Ah, do the IDs have to be specially formed? Must they be JS9-XXX, JS9-XXX_Menubar and JS9-XXX_Colorbar? Because that seems to be the only thing that makes a difference in your example.

[ericmandel]

I think I'm getting the same error now ... but js9sizes.html js9create.html works ... something silly is going on, let me look into it and get back to you, and pretty soon ...

[o-smirnov]

If I change the colorbar and menubar element IDs to use the "_Colorbar" and "_Menubar" naming schemes, at least the layout of the elements fixes itself. The window looks right. But, the Preload() call is quietly ignored, and nothing is loaded. If I use the menu to try to load a local FITS file, I get a "no FITS module available" error again, and a slew of errors in the console as described here: https://github.com/gijzelaerr/js9notebook/issues/6#issuecomment-402996941

[ericmandel]

Apparently, there are rules for naming the ids (undoubtedly, my laziness/tiredness at some point) : unique part of the name has to come first. So this works:

function newdiv() {
    id = "foo_JS9";
    // make up the html with the unique id
    var html = "<div id='foo'><div class='JS9Menubar' id='foo_JS9Menubar'></div><div class='JS9' id='foo_JS9'></div><div style='margin-top: 2px;'><div class='JS9Colorbar' id='foo_JS9Colorbar'></div></div></div>";
    // append to end of page
    $(html).appendTo($("body"));
    // create the new JS9 display, with associated plugins
    JS9.AddDivs(id);
    // just a standard load to that display
    JS9.Load("./fits/casa.fits", {colormap: "red"}, {display: id});
}

[ericmandel]

At least I documented it in https://js9.si.edu/help/webpage.html:

Multiple instances of JS9 can be added to a single web page. This is done by
giving each JS9 div element a unique ID. In order to tie otherwise separate
JS9 components together, the IDs should be specified as follows:
    start with a unique ID name, e.g. "myJS9".
    follow with an optional dash or underscore (not needed for the main window)
    finish with the type of JS9 element:"Menubar", "Panner", "Magnifier" "Info", or "Console"

[o-smirnov]

OK, that gets me somewhere... with the names just right it sort of functions (Mattias's code had led me astray...) But now there's new problems. Alas, it's getting late, so I shall have to report on these tomorrow.