search

scenerygraphics / sciview

sciview is a tool for visualization and interaction with ND image and mesh data
BSD 2-Clause "Simplified" License
68 stars 17 forks source link

Camera modes and mouse controls #209

Closed ctrueden closed 1 year ago

ctrueden commented 5 years ago

@skalarproduktraum asked me to open this issue so that we can discuss what the behavior should be for Scenery's mouse bindings.

Here is my personal taste:

Let the flaming begin!

skalarproduktraum commented 5 years ago

I moved the issue here because scenery doesn't have much of an opinion on what should be used.

kephale commented 1 year ago

🔥 hot take 🔥 we should match napari's input controls

kephale commented 1 year ago

I consider this a 1.0.0 question because if we're going to change defaults, this is probably the time to do it.

kephale commented 1 year ago

The napari mouse controls are:

kephale commented 1 year ago

Some functionality that we do have attached to mouse controls now which aren't in @ctrueden's or napari's list:

kephale commented 1 year ago

Also linking this open issue: https://github.com/scenerygraphics/sciview/issues/356

ctrueden commented 1 year ago

It sounds like napari's controls are similar to mine, so I'm :+1: for the napari behavior as defaults. I do think it's important to have a way besides mouse wheel to zoom though... otherwise how do you zoom with a touch pad? Macs have the right-edge scrolling, but I do not believe that works on all track pads. This is why I suggested shift + left drag for zoom and ctrl + left drag for pan—also because this is how VisAD worked 25 years ago, and I really liked it. But to stay consistent with napari, maybe we could make ctrl + left drag be zoom instead.

And of course with ui-behaviour's key bindings menu, users can change it how they like anyway, right? Maybe we could implement some of our favorites as presets in a dropdown combo box? I would work on this if it meant I could have a "VisAD" style that behaves exactly how I prefer! :triumph: Or maybe it's enough to just rely on ui-behaviour persisting the settings—then you only have to mess with it once per machine.

I have a related concern about the mouse behavior, though: right now when you drag the mouse, the amount that it pans or zooms or rotates is not tied intuitively to where you drag the mouse. It would be nice if e.g. when you click and drag on the object, we ray cast to pick the clicked point in XYZ space, and then when you drag, it transforms such that that same point remains beneath the mouse pointer as it moves. This is how a lot of 3D manipulation GUIs work these days, and it's very intuitive—you feel like you are gripping the thing and dragging it around. This would also avoid issues with the amount of zooming being disconnected from the current scale of the scene. Maybe this is too hard to achieve for 1.0.0, but it would make the tool feel a lot more friendly to use, IMHO.

xulman commented 1 year ago

My thoughts on this second @ctrueden ... for keybindings I would say we should go the ui-behaviour dialog (I mean that of BDV, Mastodon probably elsewhere too) window (btw, is it in sciview? I failed figuring out how to open it), which supports profiles and where/if it fails supporting mouse movements we could just complement either extending it, or with additional menu with several presets

for me, for instance, my expected controls would be similar to that of Blender... I keep regarding sciview more a 3D graphics platform than a volumetric image viewer, thus closer to Blender than to napari

xulman commented 1 year ago

but first, should we consolidate... there's something taken care of in scenery, something is added in sciview....

look, for example, I have added sciview alone (w/o scenery) into intelliJ, and (to my surprise) wasd controls are gone... the Help -> Controls dialog shows text that starts in the middle of a sentence (I recall it was a continuation of some likely-scenery-provided text), and the list of controls is halfway empty... looks like the scenery part is somehow not hooked... I don't know, I'm still trying to re-learn this whole world

xulman commented 1 year ago

My thoughts on this second @ctrueden ... for keybindings I would say we should go the ui-behaviour dialog (I mean that of BDV, Mastodon probably elsewhere too) window (btw, is it in sciview? I failed figuring out how to open it), which supports profiles and where/if it fails supporting mouse movements we could just complement either extending it, or with additional menu with several presets

for me, for instance, my expected controls would be similar to that of Blender... I keep regarding sciview more a 3D graphics platform than a volumetric image viewer, thus closer to Blender than to napari

example from my recent real-world use-case: we have a Mastodon -> Blender bridge (now trying to recreate Mastodon -> sciview one, btw, fyi), Mastodon's BDV has some moves, Blender has others, while I didn't know how to adjust Blender, I was so much happy for the ui-behaviour dialog because I could create a profile where moves where the same as in Blender

kephale commented 1 year ago

OK, so my take on the discussion is:

  1. Get ui-behavior working reliably in sciview, including ensuring that all controls can be changed within sciview.
  2. Maybe choose napari-like keybinds for the initial keybinding profile, but also consider Blender-like keybindings as another profile
kephale commented 1 year ago

Note that the keybindings UI does not seem to reflect the keybindings in Controls.kt [resolved]

kephale commented 1 year ago

We're missing support for saving keybindings to file.

kephale commented 1 year ago

update scene graph is linked to the wrong Properties refresh call [resolved]

kephale commented 1 year ago

shift + left drag = pan camera position left/right/up/down, such that the dragged point in the scene moves with the mouse

we don't account for distance between camera and drag point when calculating drag speed:

kephale commented 1 year ago

I'm starting to tweak the mouse controls a bit, and I'm appreciating @xulman's comment about sciview being closer to Blender than napari.

navigating sciview feels more like a 3D environment, which is why FPS-style controls are relevant, but napari currently doesn't actually have FPS style rotation.

xulman commented 1 year ago

actually, the FPS style is, from my own->limited experience, rather a rare way to control navigation within scene... the first day I used sciview for the first time, it took me by surprise that I am not rotating the scene like, again based on own exp., most of the SWs do

but the shift+LMB (rotate around selected object) works perfectly and nicely supplements... so feels like we have the best of both worlds, and both very easily to use/switch between

kephale commented 1 year ago

The current proposed layout is:

[removed: - ctrl + left drag = N/A]

xulman commented 1 year ago

(there's twice ctrl+left drag) (hmm, right drag (no modifiers) is moving the cam around in my case -- emulates WASD moves)

atm (on master sciview 28dd97b49e) the shift+left drag does arcball I believe and it works "opposite" for vertical mouse moves (left-right mouse move simulates rotating obj - cam moves in the opposite direction, up-down simulates cam movement)

xulman commented 1 year ago

[ this was an attempt after I misunderstood your previous post, "grayed" everything now]

to your proposal, I'm wondering what if there would exist the policy that left mouse controls the views, right mouse controls the objects... like:

  • left drag - moves cam around the fixed point (like FPS)
  • shift-left d. - could be the WASD moves (but done with the mouse, that's currently what right drag is doing) (btw, blender also pans the view with shift pressed)
  • ctrl-left d. - could do the barrel roll (so left mouse is always cam moves that don't need any obj selected etc.)
  • right drag could be the arcball (around selected object or the zoom)
  • shift/ctrl right d. could be the "normal/fast pan camera position left/right/up/down, such that the dragged point in the scene moves with the mouse" (did I get it right that you this option is moving the selected obj w.r.t. to the scene, like changing its spatial coord?) (so the right mouse is always somehow connected to the selected obj)

(I'm not insisting on anything, merely only thinking aloud)

kephale commented 1 year ago

Left click for drag

I'll point out that I meant to say that left drag is arcball rotation (which is napari's behavior)

Right click for zoom

napari uses right drag for zoom, and there was a request to make zooming easier when using trackpads.

TBH I do not love that we're using a right click drag for a 1D parameter, when we actually could be controlling 2D parameters. However, that is needed to address the trackpad usability requirement. I would be open to using a modifier for that behavior and assigning a more valuable control to right click drag.

kephale commented 1 year ago

I would really love to see a more comprehensive comparison of controls in related tools.

I was just checking Blender, and scene navigation is based on scroll + modifiers and left/right clicks are functional (left for selection and right for context menus).

xulman commented 1 year ago

note: scrolling is easy on most touchpads, scroll button (like pressing the wheel on the real mouse device) may be difficult on some touchpads (I wouldn't know how to do it on my dell latitude+linux if there wouldn't be physical left,middle,right buttons just above the touchpad), which is why I tend to avoid using it -- on the other hand if the monster blender community is keeping it, must be a signal that my worry above is really void/wrong

xulman commented 1 year ago

Left click for drag I'll point out that I meant to say that left drag is arcball rotation (which is napari's behavior)

you are right @kephale just plain LMB+drag for arcball would be the most intuitive, not only because of napari... I think I wrote somewhere earlier that FPS surprised me when I realized this is the default cam move in sciview

so if you're thinking of switching this, I'm okay with that (if I may say)

(it would anyway be the best to have a ControlsPref singleton in sciview in which user could choose if the default is arcball or FPS, flip y-axis in acrball yes/no, mouse sensitivity....)

xulman commented 1 year ago

pan camera position left/right/up/down, such that the dragged point in the scene moves with the mouse

aargh, I think I finally understand what this means :-)... it's really like dragging the scene and the scene is not modified (I understood it before that you drag the selected obj in the scene and thus modify the scene respective the obj's spatial)

in this light :-), your proposal for left-mouse (arcball supplied with panning the scene slow/fast) I find very okay :-) (thanks for carefully explaining me everything); plus right mouse with modifier does FPS or barell - also good, esp that it is with modifiers (prevents from accidental cam/sight move)

right drag (no modifiers) = [currently does arcball but i am trying to get it to zoom in/out of fixed point. it seems buggy] napari uses right drag for zoom, and there was a request to make zooming easier when using trackpads.

The current right+drag when you move mouse up/down, doesn't that feel like zooming to/out the scene? could that maybe be reused?

xulman commented 1 year ago

TBH I do not love that we're using a right click drag for a 1D parameter, when we actually could be controlling 2D parameters. However, that is needed to address the trackpad usability requirement. I would be open to using a modifier for that behavior and assigning a more valuable control to right click drag.

hmm... maybe we do the Mastodon trick, take any keyboard key and pretend it's a mouse button... like, e.g., F (press it and keep holding) + mouse move does something.... I mean to say we can do more than just shift/ctrl/LMB/RMB...

and, as you say, reserve RMB+drag for e.g. the zooming...

xulman commented 1 year ago

I would really love to see a more comprehensive comparison of controls in related tools.

another kid on the block is the BDV (and its derivates.... e.g. Mastodon, Labkit, MoBIE, BigStitcher) and its controls

since sciview and bdv are often found in Fiji :-), maybe their userbases are closest to each other... I'm thinking maybe sciview controls should be closest to that of bdv rather than to napari or blender

yet another reason to come up with control profiles, no?

kephale commented 1 year ago

Yes, I actually didn't mean to ignore the control profiles suggestion. I think we should have control profiles.

I see 2 things needing to happen:

  1. Update the default controls for sciview
  2. Add support for control profiles

1 is easy in my :eyes: because it is just twiddling with the ui-behaviour InputTriggers. 2 requires more code change and is a new feature.

Also, for 2 I was thinking there might be some related tasks like saving keybindings control to something other than CSV.

Unless someone is going to take on 2, then I would create a new issue to track it as an enhancement but maybe not make it a requirement for the 1.0.0 release.

kephale commented 1 year ago

I would like to propose that for the sake of 1.0.0 we:

smlpt commented 1 year ago

Regarding trackpad controls in Blender (as mentioned by @xulman ), they updated the behavior one or two versions ago. Now, two finger drag will rotate the camera, two finger pinching will zoom, and shift + two finger drag will pan. There is also an option to emulate the third mouse button with Alt + left click (which would then rotate), but I never use that.

As I am most accustomed to using Blender with a mouse, I will also point out that arcball rotation should have the option to change the point around which it rotates. In Blender, this center point is changed by panning and object selection (and maybe also the zoom level). The current sciview behavior is confusing because whenever you pan away from the center and then rotate using arcball, it snaps back to the center. I think it would also be fine to select the center by using mouse click + ray cast, but that might slow down the viewport interaction because the user always has to think about where to click before navigating (e.g. for me in Blender, navigation is something intuitive that I don't think about). Maybe the best approach here would be to have the center point of the arcball always be a fixed distance away from the center of the camera. Or maybe the distance should adapt to the zoom level, but that would have to be tested.