implement %%auto for cocalc-jupyter

[williamstein]

Implement a way to automatically run cells in a Jupyter notebook when the notebook is opened. This is %auto in Sage notebook and Sage worksheets, and will be needed to make a 100% compatible Sage worksheet mode for cocalc-jupyter.

The bizarre thing is that, as far as I can tell, official Jupyter doesn't have any notion of %%auto or even a proposal about it. I find this puzzling, since it was one of the first things that we implemented in Sage Notebook, and it's always been very important central functionality. Not having it doesn't make any sense to me, so maybe I'm missing something? I'm a little worried about adding something that will make cocalc-jupyter notebooks more difficult to export to non-cocalc environments, so it would be nice if we could at least make a feature proposal to jupyter classic and jupyter lab to implement the same thing... @jasongrout any thoughts?

One possibility to make this less painful might be to edit cell metadata to make things run automatically rather than explicitly using a cell magic like %%auto. Thus this would be similar to locking or hiding cells now, and auto execution would just get automatically ignored by clients that don't support this metadata.

[haraldschilly]

what are initialization cells?

if that's what I think it is, we could make this metadata-toggle a bit more discoverable in the UI, similar to readonly cells, and of course support its functionality.

[williamstein]

what are initialization cells?

Thanks for tracking that down. It looks like an "unofficial jupyter extension" I've never heard of and nobody ever requested that does what we want. However, I would -- for community compatibility -- definitely prefer to copy their metadata format and UI instead of inventing something new!

[jdchristensen]

Are there still plans to implement something like %auto for jupyter notebooks on CoCalc? I find this very useful when teaching with CoCalc, and I have a class starting May 1.

As one example, while teaching I like all unassigned values to be displayed in the output cell, without needing to call print, so I put the following at the top of my jupyter notebooks:

  # Show all unassigned values, not just the last one:
  from IPython.core.interactiveshell import InteractiveShell
  InteractiveShell.ast_node_interactivity = "all"

This is how sage notebooks behave; any chance of adding this as an option to jupyter notebooks, e.g. as a project-wide setting? (Or maybe there's a place I can hook into the initialization of all jupyter notebooks?)

[lahvak]

I just found this issue while searching for a way how to enable something like initialization cells in CoCalc's Jupyter notebooks. It would be great to have this extension, it would probably resolve about 70% of issues my students have.

I mostly use CoCalc in classes where most if not all students have no experience with coding. I like to structure my assignments in such a way that I first include some cells with "initialization" code, like loading libraries, setting up themes for plots, reading data from external files, defining some objects that students will later manipulate, and so on. Then I follow with some examples, and finally I give them some tasks to do. In class, we talk repeatedly about how they have to first run the cells that I provide before doing anything on their own. The instructions for the assignment that I post on our LMS again emphasize that they have to first run these cells before doing anything else. Finally, in the actual notebook, it says right above these cells: please run these cells before doing anything on your own. Still most of the students inevitably forget to run the cells, end up with bunch of undefined stuff, and get really frustrated. The new ChatGPT integration does not help with this, since it tells them things like to run "install.packages" and so on, while all they have to do is scroll up, click on some cells at the top of the notebook, and press shift-enter. The most frustrating thing is that I go through all that with someone on one assignment, and on the next assignment week later, they have the same issue! Either I am really bad at explaining this to them, or there is something about the design of Jupyter notebooks that makes this really non-obvious.

I think being able to mark some cells as "initialization cells" so that they will automatically run before anything else would make all this so much easier!

[haraldschilly]

@lahvak I forgot about this completely, thx for these comments. I saw my comment and I still think supporting cell.metadata.init_cell = false|true is a good idea. My idea for the actual logic & implementation would be this:

1. Just like there is a toggle for read-only and delete-protection of cells, there is a thirds called "initialization cell". For sake of consistency with other parts of CoCalc, I would signal this with a small :rocket: (rocket) icon – that's the same we use in the terminals for initializing a session.
2. When you open a notebook, it's the same as now, no new magic happens, no kernel starts.
3. But: when you click "run" any code cell, the kernel starts as usual. Right after the kernel becomes alive, all init cells are run in the order given by the notebook and errors are ignored. Then, the cell you request to run is evaluated.

The only drawback I can think of are errors of init cells that are off the screen. We could popup a warning banner if that happens.

Finally, for nbgrader support, the same logic needs to be implemented, to make the experience consistent.

[lahvak]

This is pretty much how I imagined this should work. That way the cells would run even after a kernel died, or on "Restart and run all", not just when the notebook gets open.

[williamstein]

I'm closing this because I think this is not a good idea. It's fraught with issues.