Transitioning reveal_converter to nbviewer

[damianavila]

I am trying to get notes in my nbviewer branch but in the current implementations of speaker notes I have to write a custom html and js files when the converter is called... but heroku do not have write privileges... ok, I have read that you can write a tmp file but I have to deal with that... @Carreau Do you have any insight or recommendation, or other approach to attack this problem

[Carreau]

Ha ! Now you do understand why I'm strongly opposed as using files in nbconvert ! I'll try to get a look on how does the speaker note works to understand how to get around.

But Imho nbconvert should only give you an array of potential "files" and where they wish to go, and you deal with writing them on disk if you wish. otherwise you can store in db, in cache... etc.

It will be really hepful if you can push with me for a change of API on nbconvert not to deal with file internally.

[damianavila]

El 03/01/13 10:38, Bussonnier Matthias escribió:

Ha ! Now you do understand why I'm strongly opposed as using files in nbconvert !

You are not alone, Luke... hehe

I'll try to get a look on how does the speaker note works to understand how to get around.

But Imho nbconvert should only give you an array of potential "files" and where they wish to go, and you deal with writing them on disk if you wish. otherwise you can store in db, in cache... etc.

I can have speaker notes as a potential "file", but i do not know too much about heroku apps... but it seems that I will have to read about that... One idea would be to render the speaker notes but keep it in the "dark"... I mean... when I try to convert a ipynb with slides... nbviewer would have to render the slideshow itself and, also the speaker notes html in another URL...

It will be really hepful if you can push with me for a change of API on nbconvert not to deal with file internally.

I will push with you... I have plans to deal with nbconvert in a deeper way when I finish my PhD (a few months...), but I am impatient, so I am here... annoying, hehe...

[Carreau]

Actually "notes.html " do not need to be served from the plugin/location. It seems to me that there is no server needed to synchronie the slides.

I would move it (as well as notes.js) under the "static" assets of nbconvert, and patch reveal.js so that the s key open the right URL.

It should be sufficient for it to work IIUC.

[Carreau]

And btw, I totally understand you, but I still have 2 years before finishing.

[damianavila]

El 03/01/13 11:15, Bussonnier Matthias escribió:

Actually "notes.html " do not need to be served from the |plugin/|location. It seems to me that there is no server needed to synchronie the slides.

I would move it (as well as notes.js) under the "static" assets of nbconvert, and patch reveal.js so that the s key open the right URL.

Yesterday, I have done it as you said... the problem is that notes.html takes an iframe to syncronize the slides and the notes... and the URL to put it in there is the URL from the slides... so notes.html has to change according to the names of the ipynb slideshow files... so, my strategy was to take a template of notes.html and insert the proper URL and get rendered as a new html... but i cannot write it to static in heroku...

[Carreau]

What about patching https://github.com/hakimel/reveal.js/blob/master/plugin/notes/notes.js#L8 to

window.open( 'plugin/notes/notes.html?master='+window.location.pathname

That you should be able to get dynamically in the notes

dynamically changing the src of https://github.com/hakimel/reveal.js/blob/master/plugin/notes/notes.html#L92

Using window.location.search that should return ?master=....

This could be a patch sent upstream...

Not 100% sure of the cross-browser ability to do this in a simple manner. Some function like splitting the url in multiple part might have to be re-coded.

[Carreau]

Do you have any link to the version you are working on so that I could have a look ?

[damianavila]

El 03/01/13 12:12, Bussonnier Matthias escribió:

What about patching https://github.com/hakimel/reveal.js/blob/master/plugin/notes/notes.js#L8 to

window.open( 'plugin/notes/notes.html?master='+window.location.pathname

That you should be able to get dynamically in the notes

dynamically changing the src of https://github.com/hakimel/reveal.js/blob/master/plugin/notes/notes.html#L92

Using |window.location.search| that should return |?master=....|

Interesting... I paid the price to do not know html and js... hehe... I will try and test it...

[damianavila]

El 03/01/13 12:21, Bussonnier Matthias escribió:

Do you have any link to the version you are working on so that I could have a look ?

— Reply to this email directly or view it on GitHub https://github.com/Carreau/nbviewer/issues/4#issuecomment-11847161.

I will sent you later... I am not at home until my night (your tomorrow, I guess)

[damianavila]

El 03/01/13 12:12, Bussonnier Matthias escribió:

Some function like splitting the url in multiple part might have to be re-coded.

I have done some hack to split the url and get the name of the ipynb file... I let you know...

[damianavila]

OK, I tested some of your advices...

First, I try to get the src of the iframe in https://github.com/hakimel/reveal.js/blob/master/plugin/notes/notes.html#L92 outside the iframe just passing and id:

    <div id="wrap-current-slide" class="slides">
        <iframe id="link1" width="1280" height="1024" id="current-slide"></iframe>
    </div>

    <div id="wrap-next-slide" class="slides">
        <iframe id="link2" width="640" height="512" id="next-slide"></iframe>
        <span>UPCOMING:</span>
    </div>
    <div id="notes"></div>

            <script>
            document.getElementById("link1").src = "http://slideshowviewer.herokuapp.com/url/www.slideviper.oquanta.info/nbcreveal/example_nb_tour.ipynb";
            document.getElementById("link2").src = "http://slideshowviewer.herokuapp.com/url/www.slideviper.oquanta.info/nbcreveal/example_nb_tour.ipynb";
            </script>

But in this way I lost the synchronicity between the main slideshow and the notes iframes...

Then I test writing dirty all the iframe with js:

    <div id="wrap-current-slide" class="slides">
            <script>
            document.write('<iframe src="http://slideshowviewer.herokuapp.com/url/www.slideviper.oquanta.info/nbcreveal/example_nb_tour.ipynb" width="1280" height="1024" id="current-slide"></iframe>');    
            </script>
    </div>

    <div id="wrap-next-slide" class="slides">
            <script>
            document.write('<iframe src="http://slideshowviewer.herokuapp.com/url/www.slideviper.oquanta.info/nbcreveal/example_nb_tour.ipynb" width="640" height="512" id="next-slide"></iframe>');    
            </script>
    <span>UPCOMING:</span>
    </div>
    <div id="notes"></div>

And with this approach I recover the synchronicity... I do not understand why (for sure I am missing something with js concepts). Now, I want to document.write() with the proper url (inicluding window.location) but with not too much success. @Carreau What do you think?

[Carreau]

And with this approach I recover the synchronicity... I do not understand why (for sure I am missing something with js concepts).

Hum, I'm not sure Either...

I think you will need to change https://github.com/hakimel/reveal.js/blob/master/plugin/notes/notes.js#L8 to

'whatever/note.html?master='+window.location.origin+window.location.pathname

(give the current url as a parameter.)

then you can use :

<script>
            document.write('<iframe src="'+window.location.search.slice(8)+'"' width="640" height="512" id="next-slide"></iframe>');    
</script>

Where 'slice(8)' is hardcoded to remove `?master=' there might be a cleaner way, but let's test if this works.

Alternatively you could play with document.referrer that might give you the page that opened the current page.

[damianavila]

El 04/01/13 09:24, Bussonnier Matthias escribió:

And with this approach I recover the synchronicity...
I do not understand why (for sure I am missing something with js
concepts).

Hum, I'm not sure Either...

I think you will need to change https://github.com/hakimel/reveal.js/blob/master/plugin/notes/notes.js#L8 to

'whatever/note.html?master='+window.location.origin+window.location.pathname

(give the current url as a parameter.)

then you can use :

Where 'slice(8)' is hardcoded to remove `?master=' there might be a cleaner way, but let's test if this works.

Alternatively you could play with |document.referrer| that might give you the page that opened the current page.

— Reply to this email directly or view it on GitHub https://github.com/Carreau/nbviewer/issues/4#issuecomment-11881332.

Great! Now we have reveal speaker notes working in nbviewer!!! Here you are: http://slideshowviewer.herokuapp.com/url/www.slideviper.oquanta.info/nbcreveal/example_nb_tour.ipynb Thanks for your help... Yesterday, I "squeezed" my head trying to get it but I know very little about js (and about html, and python.... yes, at the end I am a biochemist... he, I will try to study more about that... hehe, funny). I will clean my mess and give a clean branch to work for a future PR...

Thanks again...

Damián...

[Carreau]

Nice ! I hope we can have the change in reveal integrated upstream !

[damianavila]

Yep... I will integrate to my current PR in nbviewer and I will PR to reveal js library because it is an issue for other, for sure...

Thinking in the future... a question: what do you think is better?

• a centralized nbviewer supporting both conversion utilities, maybe a main page with 2 button instead of "go!", one fore html conversion and other to slideshow... or
• two twin sites, one for nbviewer and another for slideshows each one containing your own kind of conversion...

[Carreau]

I would go for 2 twin site nothing prevent from having cross references with 2 go buttons. In any case we have to wait for the metadata UI to land.

[damianavila]

El 04/01/13 12:40, Bussonnier Matthias escribió:

I would go for 2 twin site nothing prevent from having cross references with 2 go buttons. In any case we have to wait for the metadata UI to land.

— Reply to this email directly or view it on GitHub https://github.com/Carreau/nbviewer/issues/4#issuecomment-11887038.

I agree, just wondering... thanks.

[damianavila]

Just for your information, I had to change |window.location.origin| with |window.location.protocol + "//" + window.location.host| because |origin| is only available for chrome...

[Carreau]

Make sens.

was window.location.search also availlable everywhere ?

[damianavila]

El 05/01/13 07:03, Bussonnier Matthias escribió:

Make sens.

was |window.location.search| also availlable everywhere ?

— Reply to this email directly or view it on GitHub https://github.com/Carreau/nbviewer/issues/4#issuecomment-11912294.

window.location.search is more generic... I have search for cross compatibility reports, but I do not found any consistent. I tested in chrome, firerox and it works... I will test it another browsers and let you know...

[Carreau]

Thinking in the future... a question: what do you think is better?

a centralized nbviewer supporting both conversion utilities, maybe a main page with 2 button instead of "go!", one fore html conversion and other to slideshow... or two twin sites, one for nbviewer and another for slideshows each one containing your own kind of conversion...

To fully answer now,

Fernando did just register nbviewer.org, so we are thinking on how to organize it.

I was thinking on a main page (nbviewer.org) that could redirect to multiple subdomain (slide.nbviewer.org) and for example (gist.nbviewer.org) ... etc.

This allows to spawn multiple instances on heroku and share the load across multiple servers. So no single point of faillure.

It is also easier to later add (api.nbviewer.org) and (pdf.nbviewer.org) ...

Also I think heroku allow apps to share memcache instances, so if we are smart we could share the cached request to raw ipynb files to speed up things.

Drawbacks are cross-domain cookies if we need them, but github do it, so it might not be a problem.

[damianavila]

El 05/01/13 17:24, Bussonnier Matthias escribió:

Thinking in the future... a question: what do you think is better?

a centralized nbviewer supporting both conversion utilities, maybe
a main page with 2 button instead of "go!", one fore html
conversion and other to slideshow... or
two twin sites, one for nbviewer and another for slideshows each
one containing your own kind of conversion...

To fully answer now,

Fernando did just register nbviewer.org, so we are thinking on how to organize it.

I was thinking on a main page (nbviewer.org) that could redirect to multiple subdomain (slide.nbviewer.org) and for example (gist.nbviewer.org) ... etc.

This allows to spawn multiple instances on heroku and share the load across multiple servers. So no single point of faillure.

It is also easier to later add (api.nbviewer.org) and (pdf.nbviewer.org) ...

Also I think heroku allow apps to share memcache instances, so if we are smart we could share the cached request to raw ipynb files to speed up things.

Drawbacks are cross-domain cookies if we need them, but github do it, so it might not be a problem.

— Reply to this email directly or view it on GitHub https://github.com/Carreau/nbviewer/issues/4#issuecomment-11919331.

Great! I like the idea very much... thank you for the long and descriptive answer.

BTW, there is some people reporting me 404 in my instance on nbviewer... they also have the same problem with the official nbviewer... I suppose that this behavior is caused by heroku idling... searching in the web, I saw a simple solution that maybe you can use: install the free new relic add-on:

$heroku addons:add newrelic:standard

It has an availability monitor feature that will ping your site twice per minute, thus preventing the dyno from idling.

You also have to add:

newrelic==1.9.0.21 to requirements.txt

web: newrelic-admin run-program python app.py in Procfile, to initialize the agent...

Hope it helps...

Damián.

[damianavila]

I forgot... you have to set up the monitor in new relic...

Just go to the new relic:

$ heroku addons:open newrelic

Then go to settings >> availability monitoring and add the url...

Regards.

Damián

[Carreau]

404 are github timing out that are cached by memcached.

It needs nbviewer refactoring.

Thanks for the newrelic pointer. I think nb viewer get enough/day not to idle.

I'll get a look.

Matthias Le 7 janv. 2013 05:13, "Damián Avila" notifications@github.com a écrit :

I forgot... you have to set up the monitor in new relic...

Just go to the new relic:

$ heroku addons:open newrelic

Then go to settings >> availability monitoring and add the url...

Regards.

Damián

— Reply to this email directly or view it on GitHubhttps://github.com/Carreau/nbviewer/issues/4#issuecomment-11940396.

[damianavila]

El 07/01/13 03:57, Bussonnier Matthias escribió:

404 are github timing out that are cached by memcached.

It needs nbviewer refactoring.

I will look into it when I have some time...

Damián.