search

MIT-CAVE / cave_static

A react based static web app to be used as a UI and data consumer for the CAVE app.
Apache License 2.0
0 stars 1 forks source link

Implement virtual keyboard overlay (modal or draggable) launchable from input fields #470

Closed luisvasq closed 3 days ago

luisvasq commented 4 months ago

@connor-makowski, logging this feature to keep track of it since you mentioned it a few weeks ago as a long-term feature.

brianzheng205 commented 1 month ago

@connor-makowski @luisvasq @mrkwanzaa, here are some libraries for virtual keyboards:

connor-makowski commented 1 month ago

We should also attach a numerical key pad to this issue.

connor-makowski commented 1 month ago

@brianzheng205 those both look pretty solid with no additional deps added. It looks like Simple Keyboard has a lot more support (updated 2 days ago vs years ago).

luisvasq commented 1 month ago

At first glance, it seems to me that Simple Keyboard would work better for us. I see they have native support for React unlike KioskBoard. I also notice some cool features like changing the layout in terms of language and supporting key customization, which would allow for a single numeric keypad for numeric input.

connor-makowski commented 1 month ago

Just a quick follow up here. I am expecting this to be launched from the input box in the props (num props = number pad and text / combobox props = full keyboard).

I think we may end up emulating the IOS keyboard and this should appear as a draggable that can be exited and opened from the settings pane.

connor-makowski commented 1 month ago

We may even want to add a button to jump between keyboard and number pad in the component in order to generalize the draggable pad.

connor-makowski commented 1 month ago

But if launched from a component, we should switch to the relevant input type on open and allow the user to switch if desired. We should think about the ui/ux on click away.

luisvasq commented 1 month ago

Makes sense to me. My first thought on the click away UX would be to tie the display/hide of the keyboard to the focus of the input prop (combobox, text, num), unless the switch on the Settings pane is on, in which case the keyboard draggable should remain on display.

mrkwanzaa commented 1 month ago

Do we want to detect touch events (vs mouse click) and only open the keyboard if a touch caused the focus? Otherwise I'm thinking we'll want an option to never open the keyboard for non-touch users.

luisvasq commented 1 month ago

Or maybe, the switch button on the Settings pane could set the on-focus behavior for displaying the keyboard, meaning it would always always appear on the respective input when focused if the setting is on, or it would be launchable from an end adornment when the setting is off or if the user closed the keyboard.

luisvasq commented 1 month ago

I agree that it'd be best to at least set the setting on or off on the Settings pane, depending on whether it's a touch device or not.

connor-makowski commented 1 month ago

I like where this is going from a UI/UX perspective. The auto launch for touch inputs is a great feature (that should be toggleable). I am thinking no matter how we frame this one, we will want a special icon somewhere in the input line for the prop to open this if it is not open.

brianzheng205 commented 2 weeks ago

@connor-makowski @luisvasq I have implemented the bulk of the keyboard and would love your feedbacks on functionality and styling.

connor-makowski commented 2 weeks ago

Looking good. For the number pad, what does enter do?

Also can you merge in changes from 2.3.0-dev into your current branch so you can test this with new variants like the incslider? You will need to pull in changes to cave_app to get this as well.

brianzheng205 commented 2 weeks ago

enter does not do anything for any of the inputs. It doesn't seem that typing enter on a physical keyboard does anything either except for a multiline text input which adds a new line, which the virtual keyboard does not do. I will go ahead and pull changes from 2.3.0-dev.

brianzheng205 commented 2 weeks ago

@connor-makowski How can I add an incslider to one of the Python scripts in cave_app to test it?

connor-makowski commented 1 week ago

If you pull the most recent cave app (2.3.0-dev) the examples are already updated with incslider examples.

connor-makowski commented 1 week ago

Regarding enter, we may just want to remove it. Any strong reasons why not?

luisvasq commented 1 week ago

Looking good!!! I agree that enter might not be necessary. However, what if we want to add a new line to a textarea variant as @brianzheng205 pointed out (although it's not currently working for this variant)? In case we decide to keep it, we could also use it as a blur / focus-out event trigger.

luisvasq commented 1 week ago

Great job handling auto-display on touchscreen devices! I noticed that the kb launcher is not currently enabled for the slider's number input and may not be necessary since we already have spin buttons. @connor-makowski should we still have it there?

keyboard-in-slider

connor-makowski commented 1 week ago

So for enter, I like the idea of unfocus (submit the input).

connor-makowski commented 1 week ago

For the slider, I would much prefer the keyboard to the spin buttons.

brianzheng205 commented 1 week ago

I replaced the Input in the Numeric Slider with a NumberInput component since I already implemented numeric inputs for NumberInput so now the keyboard can also be used for Numeric Sliders. The Incremental Slider does not come with any inputs to change the value. Would you like me to add a NumberInput component with it as well so that users can type to change the value?

luisvasq commented 1 week ago

Great, @brianzheng205!! Let's hold off on 'incslider' for now. Is it possible to add a new line to a textarea with enter or a combination of shift + enter, whichever is easier?

brianzheng205 commented 1 week ago

@luisvasq I think it should be possible because I think I can figure out a way to distinguish between a text and textarea. Would using shift + enter make more sense so that textarea can keep the ability to press enter to unfocus the input?

luisvasq commented 1 week ago

Awesome!

@luisvasq I think it should be possible because I think I can figure out a way to distinguish between a text and textarea.

Also, that's a great point. Let's keep enter for unfocus and use shift + enter for a new line.

brianzheng205 commented 1 week ago

I decided to go with new line on enter for TextArea when the keyboard is in shift mode and have the enter text turn into new line in shift mode so it's clearer when the enter button will add a new line vs unfocusing. This means users do not necessarily need to shift + enter, they just need to press enter (labeled as new line) while in shift mode (can be accessed using shift or caps).

luisvasq commented 1 week ago

Good call. :+1: I agree that the new label for the text area is clearer in shift mode. I see that caps emulates the same behavior as shift. I'm wondering if we should enable only uppercase letters when caps is "on" while keeping the rest of the keyboard unchanged. Also, I guess we might switch back to lowercase letters when both shift and caps are on. @connor-makowski, how do you feel about this approach?

connor-makowski commented 1 week ago

See PR: https://github.com/MIT-CAVE/cave_static/pull/529