Key Interaction management in libf3d is not great and should be improved

[mwestphal]

Describe the bug Currently the management of interactions is fully handled inside of the libf3d, without the possibility of customization. Moreover, somekey are not accessible as the key input from VTK is "raw", making some key unreachable by just using the keyboard on some keyboard layout.

To Reproduce Steps to reproduce the behavior:

1. Open the file using f3d --dry-run example.glb
2. press ? to show the camera information in the terminal while using a US keyboard
3. nothing happens

Expected behavior Camera information appears.

Additional context This may require some changes in VTK, unless we want to take care of converting raw input from VTK and converting them to actual user input.

Moreover, if we tackled customization, we need to make sure the cheatsheet stay synchronized.

Wdyt @jpouderoux @Meakk ?

[mwestphal]

related to #438

[Meakk]

I agree it should be improved, and customizable.

[mwestphal]

how about an API like this:

setKeyPressCallBack(mod, key, fn) ?

What I'm not sure about is how do we convert the intention of the user : I want to do X so I press Y Key to receiving actually Y systematically and reacting upon it.

I'm not even sure about the mod part. eg, on a qwerty keyboard, pressing ? is actually pressing Shift + /.

showkeys -a display it correctly:

/        47 0057 0x2f
?        63 0077 0x3f
/        47 0057 0x2f
?        63 0077 0x3f

but VTK do not provide the right keysym or keycode:

/
Slash

Shift_L
/
Slash

If we want to provide better interaction, I think we definitely needs to improve VTK to give us the correct key code. We should not have to handle keyboard layouts in the libf3d.

Wdyt @snoyer @Meakk ?

[Meakk]

The current API is virtual interactor& setKeyPressCallBack(std::function<bool(int, std::string)> callBack) and you want to replace it with virtual interactor& setKeyPressCallBack(int mod, int keycode, std::function<bool()> callBack), right?

I think it makes sense. We can save a map in libf3d (initialized on the application side?) so the user can provide his own bindings.
Regarding the mod, what about ctrl and alt? It's a bit different than shift since the result is not a different character, so I think mod is required.

[snoyer]

If the keyboard mappings are expected to be configurable at some point it may be wise (convenient at least) to bind on strings so they can be loaded from a config file.

Is there a spec of what to expect out of VTK's keyboard events?

I slapped the following code in OnKeyPress() from interactor_impl::internals:

if (rwi->GetControlKey()) std::cout << " `ctrl`";
if (rwi->GetAltKey()) std::cout << " `alt`";
if (rwi->GetShiftKey()) std::cout << " `shift`";
std::cout<< " `" << rwi->GetKeyCode() << "`";
std::cout<< " -> `" << rwi->GetKeySym() << "`" <<std::endl;

...and, running Fedora (Gnome, X11), with a Qwerty layout I get:

• Q key: q -> q
• shift+Q key: shift Q -> Q
• /? key: / -> slash
• shift+/? key: shift ? -> question
• 1! key: 1 -> 1
• shift+1! key: shift ! -> exclam

switching the OS to Azerty layout:

• 1! key (&1 azerty): & -> ampersand
• shift+1! key (&1 azerty): shift 1 -> 1

So it looks like VTK is somehow layout-aware even though the output seems a bit confused to me. But you guys don't seem to have the same experience on your end. Does VTK delegate stuff to the OS? Do we need a minimal executable to only print keyboard events and test on different systems?

[mwestphal]

But you guys don't seem to have the same experience on your end.

I suspect maybe my Window Manager is playing some tricks on me. I will check.

Does VTK delegate stuff to the OS?

I will check the code on VTK side

Do we need a minimal executable to only print keyboard events and test on different systems?

Lets hope we dont.

[mwestphal]

@snoyer running your test I do not see the same behavior on ArchLinux + AwesomeVM:

• Q key: q -> q
• shift+Q key: shift q -> Q
• /? key: / -> slash
• shift+/? key: shift / -> slash
• 1! key: 1 -> 1
• shift+1! key: shift 1 -> 1

switching the OS to Azerty layout:

• 1! key (&1 azerty): & -> ampersand
• shift+1! key (&1 azerty): shift & -> ampersand

In short, in my case, VTK is not aware of the layout at all apart from the base key, which I why I can't reach the ? key.

I will try other OSes and DE.

[mwestphal]

Do we need a minimal executable to only print keyboard events and test on different systems?

A quick test is to press the Shift + /? key with F3D binary, which show an output in the terminal. If it doesn't show anything with a QWERTY keyboard, then it means VTK does not pass the expected keysym.

[mwestphal]

Tested on Ubuntu 22, MacOS and Windows, they all behave like yours @snoyer .

[mwestphal]

Note: VTK has been fixed, see here for more info: https://discourse.vtk.org/t/about-keycode-keysym-modifiers-crossplatform-and-reliable-code/12598

[mwestphal]

WIll need a libf3d options rework