search

napari / napari

napari: a fast, interactive, multi-dimensional image viewer for python
https://napari.org
BSD 3-Clause "New" or "Revised" License
2.09k stars 411 forks source link

How to handle headless app-model actions #6917

Open lucyleeow opened 2 weeks ago

lucyleeow commented 2 weeks ago

This is a request for comments about how to handle app-model actions in headless mode (i.e. without a Qt Viewer). There have been discussions here and there but nothing well documented and no concrete decisions. Recent PRs have brought up this topic and this is an attempt at gaining more clarity.

Originally, there was separation of actions/providers/processors (actions for short) into 'GUI/Qt' and 'non-GUI/Qt'. But it seems that most of the actions are technically Qt (see #6767, #6883, #6743)

Considerations about how to separate 'GUI/Qt' and 'non-GUI/Qt' action files:

The aim of separating 'Qt' and 'non-Qt' actions is such that non-Qt actions are registered and available in the app headless. But this brings more considerations:

  1. Which built-in actions should be made available to the headless app? i. I think all (?) the menubar actions that might be useful headless, already have a public API function (e.g., open_sample) ii. Some layer actions would make sense headless but see 2.
  2. Currently there are no 'non-Qt' providers. This means that the layer actions which technically do not need Qt, won't work without Qt (the layerlist provider uses _viewer_provider which gets the active viewer via _QtMainWindow) i. when does headless dependency injection make sense? When headless we do not know which viewer/layer is active (discussed more here #6768). ii. if headless does not use dependency injection, does that mean GUI actions that do reply on dep injection could not be used via CommandRegistry.execute_command ? What do we want headless action API to look like?

cc @DragaDoncila @jni @psobolewskiPhD @Czaki who may be interested. Thank you

DragaDoncila commented 1 week ago

We think that actions that do not technically require QT but are defined for the sole purpose of living in a menu, should be considered a 'GUI' action.

I think this bit here is very tricky because it's a a consideration on how people want to use napari. For example, take any given Open sample action that currently lives in the file menu. This is obviously not only a GUI action - people want to open files in headless mode too! However, there's already a blessed API for that that has nothing to do with the app-model file menu open sample action. Would a user ever want to trigger the menu version of the open sample action in headless mode? I think the answer is no. Does that mean the file menu actions for opening samples are GUI actions? Lucy and I have basically decided yes. This also means the action is not registered until all other GUI actions are registered. We think this makes sense - we don't want actions registered with the app that are guaranteed to fail right?

However, there is then the further distinction of GUI vs. Qt. Is the File -> Open Sample action qt-specific? No, probably not. Any frontend we used would likely include some way of defining menubar actions. Where should this action live then, structurally speaking, given that we definitely don't want it to be registered and available for execution in headless mode, but it's also not specifically Qt dependent?

On the topic of dependency injection in headless mode, I have no real idea whether it makes sense or not, or what parameters users might request that they can't explicitly pass in but will be available at runtime. The current providers for viewer and layers work on the basis that through the GUI, you can infer which viewer/layer the user wanted associated with a particular action e.g. because a certain viewer has focus or because a certain layer is active and selected in the layerlist. Should we (and can we even) make those same inferences in headless mode? It doesn't seem natural...

lucyleeow commented 1 week ago

For example, take any given Open sample action that currently lives in the file menu.

I think that falls under:

I think all (?) the menubar actions that might be useful headless, already have a public API function (e.g., open_sample)

But I should have been more specific in the kind of actions I was talking about there. E.g., the help menu actions that open up a URL.

given that we definitely don't want it to be registered and available for execution in headless mode, but it's also not specifically Qt dependent?

I think technically the open sample one is Qt dependent because we use handle_gui_reading which uses Qt. But this does raise a good question about functions which get a 'wrapper' for actions that get added to the menu. In this case we have the open_sample API which users can use directly. Potentially there would be other actions where the menu version gets a wrapper which is not needed headless? In this case do we have 2 actions??

Where should this action live then, structurally speaking, given that we definitely don't want it to be registered and available for execution in headless mode, but it's also not specifically Qt dependent?

I feel like this question is out of scope of this issue. It's a long standing issue of napari, that there isn't a GUI but not Qt folder? It's touched upon here: https://github.com/napari/napari/issues/6469

psobolewskiPhD commented 1 week ago

Question: am I misunderstanding or there should be a distinction between headless and programmatic? I personally don't see much value (or have interest in) the first, but the 2nd, programmatic access to things in an viewer, is something that is absolute gold and we should strive for 1:1 equivalence as much as possible.

Edit: to expand, while something like napari.help() or whatever that just opens some URLs to docs may be a bit weak, I think it's not useless! Especially in a jupyter notebook care where they are already using a browser.

lucyleeow commented 1 week ago

Sorry I think I mean headless.

i.e., init_qactions (which registers all the Qt actions) gets run when we init _QtMainWindow.

So whenever there is a Qt Viewer (vs ViewerModel) all the Qt actions and such are registered. So they are available for use.

Note also the other question is how we want the user to trigger the actions. This is currently possible via their command ID string, but this is a bit long (and we are not exactly sure how to name it... see https://github.com/napari/docs/pull/405). E.g., for the help actions the strings are currently e.g., 'napari.window.help.info'. I would think that a different public API to accessing such actions would be nicer (e.g., napari.help) (But would need example of actions/use cases to better determine this)