Support for Global Variables / Notebook Parameters

[okennedy]

TL;DR: Allow notebooks to declare single-valued parameters instead of needing to create single-cell datasets.

A frequent request at SIGMOD was for notebook parameters. The idea is to abstract out specific workflow configuration elements into a separate area of the notebook. These parameters would then be accessible in individual cells. For example:

• Python and Scala cells could use the vizierdb object (e.g., vizierdb['myparameter']) or similar.
• SQL cells could use JDBC parameters (e.g., SELECT * FROM foo WHERE bar = ?myparameter) or similar
• PLOT cells culd use something like SQL cells (i.e., ?myparameter) or possibly a pop-up menu for column selections
• LOAD DATA cells could take a URL or uploaded file as a parameter.

The primary benefit of parameters is that it becomes possible to export the notebook functionality as a library. To allow external notebooks to call in to this notebook, you need to have a way to communicate configuration options from outside -- parameters are a natural way to do this.

There are a few secondary benefits

• It makes it easier to isolate and track the provenance of specific configuration parameters.
• It makes it easier to pass common state between cells (e.g., instead of creating single-cell datasets).
• It puts configuration parameters and other magic values front and center, making it easier to change major features of the workflow in one place. (as an aside, these parameters would be overridden when the notebook is exported as a library --- this is great for development!)
• It could, in principle, enable better integration with interactive visualization tools like Bokeh. Bokeh has a server mode that allows the frontend to pass back parameters to the code generating the visualization. We could track the provenance of such parameters, and allow those parameters to be re-used in subsequent cells.

Parameters could be typed: including native types (int, float, string, date, etc...), as well as workflow-specific types (column-set, dataset, etc...). That in turn lets us create custom UIs for each type (e.g., file uploading for datasets).

[julianafreire]

Check https://github.com/nteract/papermill

[okennedy]

Yes, exactly this... though with a first-class treatment in the UI