search

marimo-team / marimo

A reactive notebook for Python — run reproducible experiments, execute as a script, deploy as an app, and version with git.
https://marimo.io
Apache License 2.0
7.68k stars 264 forks source link

lazy cells (re-work of disabled cells) #2100

Open ggggggggg opened 2 months ago

ggggggggg commented 2 months ago

Description

I would like the "reactive execution"/"disabled" cell status be more clear and useful, by unifying it with "lazy execution". Currently there are multiple forms of lazyness that are presented with different names, different UIs, and different feature sets. In particular "reactive execution" lacks features that "lazy" has. Here I propose that these be combined into one feature, called "lazy".

Suggested solution

Unify on term, I suggest "lazy", with one UI, and one feature set. I suggest the UI and feature set that "lazy" currently has.

Alternative

  1. Leave it unchanged, I believe the current state is a local minimum of usefulness and clarity, so this isn't a great solution.
  2. Pick a different term to unify on, "reactive execution" is fine.

Additional context

In the settings menu you can set the notebook to "autorun" or "lazy". image When in lazy mode, dependent cells of a changed cell are marked "stale" and require user interaction to run. image For each cell we can choose the "reactive execution" state. In this case the changed cell is unable to be run. image

image The user actually can run it, by enabling "reactive execution", running it, then again disabling "reactive execution". It is more ceremony, but fundamentally the same functionality as lazy. I think "reactive execution" should be replaced by a cell by cell version of "lazy". This will provide the following benefits:

  1. A single term the same functionality. I found the use of different terms misleading, as it suggested these features were fundamentally very different. So I made guesses about how they worked that were wrong. We can prevent this mistake for future users.
  2. More functionality with based on cell by cell toggle. Allowing opt in cell by cell lazy makes it easy to break a notebook into chunks to separate out slow or annoying parts while working on an earlier part. There is a "run button" which is documented to provide a similar functionality, so it's clear this feature is useful.
akshayka commented 2 months ago

I always like having as few concepts as possible. In this case I view disabling cells as different than making them as lazy — disabled cells and their descendants never run, whereas a proposed "lazy" cell could still be executable.

The advantage of disabled functioning as it does is that you can still iterate on the rest of your notebook, even using the run all stale cells button, while having the confidence that your disabled cells won't run, not under any circumstance. For example, you may want to disable a cell from running if it writes to a production database or has some other side-effect.

Does this make sense? If so, do you have any suggestions on how we can make this behavior clearer?

ggggggggg commented 2 months ago

I see, so there is another use case. I was unaware of the run all stale cells button. Perhaps a 3 way toggle then.

  1. Reactive
  2. Lazy (or Manual)
  3. Protected

For the third case if it's really meant to isolate critical parts like writing to a production database, I'd expect a stronger and more explicit UI element that expresses intention. Like you have to click on a button that says "warning: will write to production database on click". Maybe it's a "protected cell" and you have to type "yes I mean it" to get it to run. Or maybe it's built with the other UI components available, like a toggle switch on the cell to enable/disable one code path that writes to production.

Also, where is "run all stale cells?" I don't see it even when I switch to lazy execution, nor can I find it in the command pallet. Do you mean the "run all modified cells" button?

akshayka commented 2 months ago

Perhaps a 3 way toggle then ... For the third case if it's really meant to isolate critical parts like writing to a production database, I'd expect a stronger and more explicit UI element that expresses intention.

That's interesting. "Protected" would be a more relaxed form of disabled. I will consider, but I wonder if it's more confusing having even more options? ie a 3-way toggle vs two. That said I know what we have today confused you, so it seems there's ways for us to improve.

Also, where is "run all stale cells?" I don't see it even when I switch to lazy execution, nor can I find it in the command pallet. Do you mean the "run all modified cells" button?

Yes, that's what I meant. Sorry for misspeaking; perhaps we should update the tooltip to be "run all stale cells". It's the button in the bottom right of the notebook that appears when at least one cell is stale (due to lazy execution or due to modification).

ggggggggg commented 2 months ago

That's interesting. "Protected" would be a more relaxed form of disabled.

My intention of suggesting a name like "protected" isn't to be more relaxed, it is to more clearly express intention. Compare the suggested user interaction required to run a cell, and leave it in a non-reactive state. Protected: select protected box, type a short phrase and press a button Disabled: cell menu, enable reactive execution, ctrl-enter, cell menu, disable reactive execution

I don't see one of those as significantly more difficult or relaxed than the other. Instead I prefer "protected" because it would

  1. naturally leave the cell in the same state after a single execution
  2. Answer's a users potential question "why is this cell different from normal lazy cell?"

I will consider, but I wonder if it's more confusing having even more options? ie a 3-way toggle vs two. That said I know what we have today confused you, so it seems there's ways for us to improve.

There are already 3 options for how cells behave. Reactive, lazy, and disabled. I am suggesting unifying them into one UI to make their relationship more clear. I'm not tied to a 3-way toggle, but it seems pretty natural for a 3 option system.

Yes, that's what I meant. Sorry for misspeaking; perhaps we should update the tooltip to be "run all stale cells". It's the button in the bottom right of the notebook that appears when at least one cell is stale (due to lazy execution or due to modification).

I'm experimenting with running the whole notebook in lazy mode so I now see this button, soon I will have more insight into the advantages of the current design.

akshayka commented 2 months ago

Answer's a users potential question "why is this cell different from normal lazy cell?"

This is a good point, thanks for emphasizing it.

Protected: select protected box, type a short phrase and press a button Disabled: cell menu, enable reactive execution, ctrl-enter, cell menu, disable reactive execution

When a cell is disabled, the user can edit a notebook and add it to the graph without running it — the run button is replaced with an "add to graph" button when a cell is disabled. This allows the user to persist changes to the graph without running the cell, and is a functionality that some of our users requested when we designed this feature. When the cell is re-enabled, if it has been made stale by the execution of other cells, it will automatically run (if "on cell change" is set to autorun) or marked as stale (if "on cell change" is set to "lazy").

If we decided to replace "disabled" with "protected", we might want to to allow both running (with confirmation) and adding to the graph without running. This could perhaps be done by adding another button, or by a click on a cell's "run" button giving two options — actually run the cell, or add to the graph.

Thanks for the feedback, I will continue mulling it over and please feel free to share additional thoughts as they come.