ipython-contrib / jupyter_contrib_nbextensions

A collection of various notebook extensions for Jupyter
http://jupyter-contrib-nbextensions.readthedocs.io/en/latest
Other
5.22k stars 806 forks source link

All extensions don't work with big notebooks. Load timeout issue #1195

Open ODemidenko opened 6 years ago

ODemidenko commented 6 years ago

The version of the notebook server is 5.0.0 and is running on: Python 3.6.2 |Anaconda custom (64-bit)| (default, Jul 20 2017, 12:30:02) [MSC v.1900 64 bit (AMD64)] browser: Chrome

With small/new notebooks - all extensions work well. With a big notebook - they don't start. I've tried setTimeout and enabling/disabling Highlighter extension as was suggested in issue 1009 But that helped only once (and I didn't get which advice actually helped). And now the problem continues.

In the JS console I see following error:

main.min.js:12431 Failed to load extensions: Array(16) Error: Load timeout for modules: 
custom/custom,nbextensions/nbextensions_configurator/config_menu/main,nbextensions/contrib_nbextensions_help_item/main,nbextensions/collapsible_headings/main,nbextensions/toc2/main,nbextensions/varInspector/main,nbextensions/snippets_menu/main,nbextensions/hide_input_all/main,nbextensions/highlighter/highlighter,nbextensions/highlight_selected_word/main,nbextensions/execute_time/ExecuteTime,nbextensions/skip-traceback/main,nbextensions/freeze/main,nbextensions/chrome-clipboard/main,nbextensions/move_selected_cells/main,nbextensions/init_cell/main
http://requirejs.org/docs/errors.html#timeout
    at makeError (require.js:165)
    at checkLoaded (require.js:693)
    at require.js:714

probably I used setTimeout option wrong? Am I right that it should be added to notebook metadata, like this:

{
"setTimeout": 60,
  "hide_input": false,
  "kernelspec": {
    "name": "python3",
    "display_name": "Python 3",
    "language": "python"
  },
  "language_info": {...

? BTW: I have just updated nbextensions and this didn't help either

jcb91 commented 6 years ago

With small/new notebooks - all extensions work well. With a big notebook - they don't start.

Unfortunately, although this (timeout) can be caused by nbextensions which take a long time to load, it's also an issue with notebook itself. You can check https://github.com/jupyter/notebook/issues/2075 for details, but essentially as I understand it, what's happening is that

  1. notebook javascript fires off requests to load nbextensions
  2. notebook javascript starts loading all the notebook content (rendering markdown cells etc)
  3. loading big notebooks takes a long time and, crucially, it blocks the loading of require-specified scripts from 1. As a result, the require library assumes the requests have timed out, and doesn't bother to try loading the nbextensions.

probably I used setTimeout option wrong? Am I right that it should be added to notebook metadata, like this:

To mitigate this issue, you can increase the timeout for requirejs as specified in https://github.com/ipython-contrib/jupyter_contrib_nbextensions/issues/822#issuecomment-269912319. But, this doesn't go in notebook metadata, or notebook config. Instead, you need to add something like the following in your custom.js:

// default is 30s, increase to 1 minute
window.requirejs.config({waitseconds: 60});

You can find details of where to find/create a custom.js file at http://jupyter-notebook.readthedocs.io/en/latest/examples/Notebook/JavaScript%20Notebook%20Extensions.html#custom.js. This (using custom.js instead of an nbextension) is necessary because it's the only way to guarantee that it gets executed before the notebook load call.

Alternatively, in a strategy that's only just occurred to me (so is untested!) you could delay the (attempted) loading of nbextensions until after the notebook fully loads, using the following in custom.js:

require(['base/js/namespace', 'base/js/utils', 'base/js/events'], function (Jupyter, utils, events) {
    "use strict";
    // only bother if we're in notebook view
    if (Jupyter.notebook === undefined) return;
    // patch load_extensions to delay until after the notebook has fully loaded
    var orig_load_extensions = utils.load_extensions;
    utils.load_extensions = function () {
        var nbexts = arguments;
        if (Jupyter.notebook._fully_loaded) return orig_load_extensions.apply(this, nbexts);
        return new Promise(function (resolve, reject) {
            console.log('deferring loading of nbextensions');
            events.one('notebook_loaded.Notebook', function () {
                console.log('loading deferred nbextensions');
                resolve(orig_load_extensions.apply(this, nbexts));
            });
        });
    };
});
ODemidenko commented 6 years ago

@jcb91 Thank you! It helped! (first approach)

webermarcolivier commented 5 years ago

I was having the same problem, with very large notebook, the TOC2 extensions failed to load, in Firefox, jupyter notebook 5.5.0, jupyter_contrib_nbextensions 0.5.0. Increasing the timeout in javascript custom.js worked fine. The notebook needed about 90 seconds to fully load.

Just to mention that there is a typo, the correct attribute is waitSeconds, not waitseconds, such that the line to add in custom.js is

window.requirejs.config({waitSeconds: 120});
cholmcc commented 5 years ago

Hi all,

I also have a very large NoteBook - more than 1k cells - many of which most are Markdown (~90%) and quite heavy on MathJax. For this NB, I also had trouble getting various extensions to work - loading would timeout as described above. The extensions I was loading were

I'm using stock Jupyter from Debian sid and Google Chrome (unstable).

For me none of the suggestions worked. That is putting

// default is 30s, increase to 1 minute
window.requirejs.config({waitseconds: 60});

(or with even larger values) into ~/.jupyter/custom/custom.js didn't resolve the issue. The trouble seem to me that in main.min.js rendering of the NB cells is started immediately after custom/custom and other extensions have been scheduled for loading. Since there are a great number of cells requestjs hardly ever have a chance of executing the queued loads, and by the time custom.js is executed it is too late.

My solution to the problem was inspired by #822 - that is, I add a cell with the content

%%javascript
/** Below because this notebook is pretty large */
var rto = 120;
console.log('NB: Increase require timeout to ' + rto + ' seconds');
window.requirejs.config({waitSeconds: rto}); 

to the top of the NB. From the log I see that this code is executed early, and in fact, it seems to speed up the loading of the NB in general.

I don't find this to be an issue with the nbextensions - rather it seems a "bug" in notebook/base/js/main.js. Perhaps it should be reported upstream (if not already done). Perhaps main.js should have heuristics to modify the requestjs.config(waitSeconds:) according to the size (in cells) of the NB. Or, perhaps one should be able to set the timeout per NB in the metadata.

It would really be a shame if people went to MatLab or Mathematica because of this.

BTW, it seems that the extension nbTranslate has heuristics to postpone loading until the NB is fully done. Perhaps one can address that issue (i.e., deferred set-up of extensions) in a similar maner for other extensions.

Just my 2€¢

Christian

Benfeitas commented 5 years ago

I'm using stock Jupyter from Debian sid and Google Chrome (unstable).

For me none of the suggestions worked. That is putting

// default is 30s, increase to 1 minute
window.requirejs.config({waitseconds: 60});

(or with even larger values) into ~/.jupyter/custom/custom.js didn't resolve the issue. The trouble seem to me that in main.min.js rendering of the NB cells is started immediately after custom/custom and other extensions have been scheduled for loading. Since there are a great number of cells requestjs hardly ever have a chance of executing the queued loads, and by the time custom.js is executed it is too late.

My solution to the problem was inspired by #822 - that is, I add a cell with the content

%%javascript
/** Below because this notebook is pretty large */
var rto = 120;
console.log('NB: Increase require timeout to ' + rto + ' seconds');
window.requirejs.config({waitSeconds: rto}); 

to the top of the NB. From the log I see that this code is executed early, and in fact, it seems to speed up the loading of the NB in general.

In my case, I had this same problem running Firefox in CentOS. I have tested the window.requirejs.config({waitSeconds: 120}); (which worked), but this solution seems to load the notebook faster.

iromeo commented 4 years ago

@jcb91

Alternatively, in a strategy that's only just occurred to me (so is untested!) you could delay the https://github.com/ipython-contrib/jupyter_contrib_nbextensions/issues/1195#issuecomment-357756819

Thx, works for me

Artyrm commented 3 years ago

Hi. With recent install of jupyter core : 4.6.3 jupyter-notebook : 6.0.3

extensions load and fail (as seen in js console) happens before custom.js gets loaded (I judge it from printing to console at the beginning od custom.js) . Hence, no workarounds suggested here worked. Are there any other ways?

Timmate commented 3 years ago

Hi all,

I also have a very large NoteBook - more than 1k cells - many of which most are Markdown (~90%) and quite heavy on MathJax. For this NB, I also had trouble getting various extensions to work - loading would timeout as described above. The extensions I was loading were

  • nbTranslate
  • toc2
  • hinterland
  • jupyter-js-widgets

I'm using stock Jupyter from Debian sid and Google Chrome (unstable).

For me none of the suggestions worked. That is putting

// default is 30s, increase to 1 minute
window.requirejs.config({waitseconds: 60});

(or with even larger values) into ~/.jupyter/custom/custom.js didn't resolve the issue. The trouble seem to me that in main.min.js rendering of the NB cells is started immediately after custom/custom and other extensions have been scheduled for loading. Since there are a great number of cells requestjs hardly ever have a chance of executing the queued loads, and by the time custom.js is executed it is too late.

My solution to the problem was inspired by #822 - that is, I add a cell with the content

%%javascript
/** Below because this notebook is pretty large */
var rto = 120;
console.log('NB: Increase require timeout to ' + rto + ' seconds');
window.requirejs.config({waitSeconds: rto}); 

to the top of the NB. From the log I see that this code is executed early, and in fact, it seems to speed up the loading of the NB in general.

I don't find this to be an issue with the nbextensions - rather it seems a "bug" in notebook/base/js/main.js. Perhaps it should be reported upstream (if not already done). Perhaps main.js should have heuristics to modify the requestjs.config(waitSeconds:) according to the size (in cells) of the NB. Or, perhaps one should be able to set the timeout per NB in the metadata.

It would really be a shame if people went to MatLab or Mathematica because of this.

BTW, it seems that the extension nbTranslate has heuristics to postpone loading until the NB is fully done. Perhaps one can address that issue (i.e., deferred set-up of extensions) in a similar maner for other extensions.

Just my 2€¢

Christian

Thank you a lot! Had the same problems with Jupyter Notebook v6.1.5, Chromium v87.0.4280.88. Adding that javascript cell, running it and reloading the notebook resolved the issue.

ibobak commented 2 years ago

I've found that adding

window.requirejs.config({waitSeconds: 120});    

won't help: due to unknown reason in my environment some other code (which code - I did not found) is forcibly setting it back to 30.

This is how I found this: I opened the file /opt/anaconda3/lib/python3.8/site-packages/notebook/static/components/requirejs/require.js and added some logging statements:

console.log("checkLoaded", waitInterval, expired, noLoads, reqCalls, stillLoading, needCycleCheck);

and to my surprise waitInterval was equal to 30000 no matter that I've set waitSeconds to 0, then to 180, then to 240 - whatever.

So the BRUTE FORCE patch which I did looks like this: I've set waitInterval to be equal to 0:

        function checkLoaded() {
               var countOfTrying = 0;
            var err, usingPathFallback,

                waitInterval = 0,

                //It is possible to disable the wait interval by using waitSeconds of 0.
                expired = waitInterval && (context.startTime + waitInterval) < new Date().getTime(),
                noLoads = [],
                reqCalls = [],
                stillLoading = false,
                needCycleCheck = true;