search

cal-itp / data-analyses

Place for sharing quick reports, and works in progress
https://analysis.calitp.org
27 stars 6 forks source link

Bug: Timeout for executing notebooks #236

Closed tiffanychu90 closed 1 year ago

tiffanychu90 commented 2 years ago

Where did the bug occur? Select from the below, and be sure to affix the appropriate label to this issue (e.g. dataset, jupyterhub, metabase, analytics.calitp.org)

Describe the bug There seems to be an error that comes up when deploying larger notebooks to the portfolio: nbclient.exceptions.CellTimeoutError: A cell timed out while it was being executed, after 4 seconds. The message was: Timeout waiting for IOPub output.

For cells that are creating charts or maps, 4 seconds is probably way too short.

To Reproduce Steps to reproduce the behavior:

  1. Go to this commit
  2. In terminal (cd in data-analyses): make build_highway_corridors will show this time-out.

Expected behavior We want the notebook to fully execute so the Jupyterbook can build. This behavior would likely affect all kinds of deployment (though in this case, it's an unparameterized notebook). We want to set a reasonable timeout to allow cells executing charts or maps to render, but also prevent super large files from being created.

Additional context We might think of a fallback for unparameterized notebook that relies on jupyter nbconvert and writes directly to HTML, but have a way to still link that to portfolio?

tiffanychu90 commented 1 year ago

Timeout has been bumped? Hasn't been an issue as much when data processing is done in a script.