How to inject a VirtualKeyboard

[djonasdev]

I have made simple VirtualKeyboard.

In my case I would like to as soon as the TextBox receives the InputFocus, transfer the Text of the TextBox to the VirtualKeyboard and open it in a separate Window (RaspberryPie currently only supports a single window). This is finally closed with Return and transferred to the TextBox.

I would like to avoid having to create my own CustomTextbox that the VirtualKeyboard opens automatically.

Now I would like to combine it with avaloniaui. As you already mentioned here https://github.com/AvaloniaUI/Avalonia/issues/3415 it should be possible. But actually I'm a little bit stuck and don't know how to go on.

The events TextInputOptionsQuery and TextInputMethodClientRequested are not called.

var boxes = this.GetLogicalDescendants().OfType<TextBox>().ToList();
foreach (var box in boxes)
{
    box.TextInputOptionsQuery += (o, eventArgs) =>
    {
        // is not called
    };
    box.TextInputMethodClientRequested += (o, eventArgs) =>
    {
        // is not called                        
    };
}

As soon as the VirtualKeyboard works, I will share my code so that it may even be included in the repo or to improve it.

[maxkatz6]

@kekekeks can you take a look?

[kekekeks]

Right now InputMethodManager only looks for input method implementation in the visual root: https://github.com/AvaloniaUI/Avalonia/blob/43c04c2266348ded8b6698f8b497bedccb56fbf9/src/Avalonia.Input/TextInput/InputMethodManager.cs#L79-L81

We probably need to change it to traverse the visual tree to find a visual that provides input method capabilities and use that. It will require to handle some events to account for cases when said visual suddenly gets removed from the visual tree, that's why the initial implementation only uses the root.

For now you can implement your own Window base class and implement ITextInputMethodRoot there. It will enable input method interactions.

[djonasdev]

For those who also want to implement a virtual keyboard, here is the source code.

Improvements are always welcome

---

Layout/KeyboardLayout.cs

```cs public abstract class KeyboardLayout : UserControl { string LayoutName { get; } } ```

Layout/VirtualKeyboardLayoutDE.axaml

```cs public partial class VirtualKeyboardLayoutDE : KeyboardLayout { public VirtualKeyboardLayoutDE() { InitializeComponent(); } private void InitializeComponent() { AvaloniaXamlLoader.Load(this); } public string LayoutName => "de-DE"; } ``` ```xaml ```

Layout/VirtualKeyboardLayoutUS.axaml

```cs public partial class VirtualKeyboardLayoutUS : KeyboardLayout { public VirtualKeyboardLayoutUS() { InitializeComponent(); } private void InitializeComponent() { AvaloniaXamlLoader.Load(this); } public string LayoutName => "en-US"; } ``` ```xaml ```

VirtualKey.axaml

```cs public class VirtualKey : TemplatedControl { public static readonly StyledProperty ButtonCommandProperty = AvaloniaProperty.Register(nameof(ButtonCommand)); public ICommand ButtonCommand { get { return GetValue(ButtonCommandProperty); } set { SetValue(ButtonCommandProperty, value); } } public static readonly StyledProperty NormalKeyProperty = AvaloniaProperty.Register(nameof(NormalKey)); public string NormalKey { get { return GetValue(NormalKeyProperty); } set { SetValue(NormalKeyProperty, value); } } public static readonly StyledProperty ShiftKeyProperty = AvaloniaProperty.Register(nameof(ShiftKey)); public string ShiftKey { get { return GetValue(ShiftKeyProperty); } set { SetValue(ShiftKeyProperty, value); } } public static readonly StyledProperty AltCtrlKeyProperty = AvaloniaProperty.Register(nameof(AltCtrlKey)); public string AltCtrlKey { get { return GetValue(AltCtrlKeyProperty); } set { SetValue(AltCtrlKeyProperty, value); } } public static readonly StyledProperty

[ichbinvinh]

Thank you for sharing your code. But there is no code for the class CoporateWindow. Can you please share this class for everyone? Thank you so much!

[djonasdev]

@ichbinvinh I added the missing CoporateWindow class. However, it is not absolutely necessary to use this.

[elveejay1]

@dojo90 I can't see the code for VirtualKey.axaml? I'm a bit new to Avalonia, so it may just be me...

Things work, but the icon buttons (shift, enter etc) does not show icons

[djonasdev]

@dojo90 I can't see the code for VirtualKey.axaml? I'm a bit new to Avalonia, so it may just be me...

Things work, but the icon buttons (shift, enter etc) does not show icons

I admit that I posted the sources a bit confusing.

I have now summarized all files in individual spoilers 😉

[elveejay1]

@dojo90 I can't see the code for VirtualKey.axaml? I'm a bit new to Avalonia, so it may just be me... Things work, but the icon buttons (shift, enter etc) does not show icons

I admit that I posted the sources a bit confusing.

I have now summarized all files in individual spoilers 😉

No problems. I think your work is super cool. Thanks for responding so fast to a very old post

I found out why I didn't see the icons. If someone else runs into the same problem as me, they can check: https://github.com/AvaloniaUtils/Material.Icons.Avalonia It turned out I needed an application style... `<Application ...>

... `

[vs-savelich]

@dojo90 did you have a chance to port this implementation to Avalonia 11?

[djonasdev]

@vs-savelich I'm sorry, but I no longer use avalonia. 😐

[nullx1337]

Hey I found a solution to refactor the code so it works under Avalonia 11.

Instead of implementing the ITextInputMethodRoot Interface on the Mainwindow, we could use GotFocusEvent and filtering it for Textbox controls. It is important to somehow clear the Focus. Currently I use the FocusManager.ClearFocus() but this method will be removed in a 11.x update. Because ProcessInput of InputManager isnt available an there are some limitations and visual bugs. for example the Cursor in the TextBox stays always a the beginning of the Textbox.

I also have a strange bug on Linux ARM. If I open the Virtualkeyboard in a Dialog the VirtualKeyboard will reappear because the focus isn't cleared and as soon the VirtualKeyboard closes, the TextBox regains focus and will open the Virtualkeyboard again.

 InputManager.Instance.ProcessInput(new RawKeyEventArgs(KeyboardDevice.Instance, (ulong)DateTime.Now.Ticks, (Window)TextBox.GetVisualRoot(), RawKeyEventType.KeyDown, key, RawInputModifiers.None))

Window

 public partial class MainWindow : Window
    {
        private VirtualKeyboardTextInputMethod virtualKeyboardTextInput = null;
        public MainWindow()
        {
            InitializeComponent();
            this.SystemDecorations = SystemDecorations.None;
            this.WindowStartupLocation = WindowStartupLocation.CenterScreen;
            virtualKeyboardTextInput = new VirtualKeyboardTextInputMethod((Window)this);

            this.AddHandler < GotFocusEventArgs>(Control.GotFocusEvent, openVirtualKeyboard);
        }

        private void openVirtualKeyboard(object? sender, GotFocusEventArgs e)
        {
            if(e.Source.GetType() == typeof(TextBox))
            {
                FocusManager.ClearFocus();
                virtualKeyboardTextInput.SetActive(true, e);

            } 
        }
}

VirtualKeyboardTextInputMethod

public class VirtualKeyboardTextInputMethod 
    {
        private bool _isOpen;
        private TextInputOptions? _textInputOptions;

        private Window root = null;

        public VirtualKeyboardTextInputMethod(Window root)
        {
            this.root = root;
        }
        public VirtualKeyboardTextInputMethod()
        {

        }

        public async Task SetActive(GotFocusEventArgs e)
        {
            if (!_isOpen )
            {

                _isOpen = true;
                var oskReturn  = await VirtualKeyboard.ShowDialog(_textInputOptions, this.root);

                if(e.Source.GetType() == typeof(TextBox))
                {
                    ((TextBox)e.Source).Text = oskReturn;

                }

                _isOpen = false;
                _textInputOptions = null;

                if (this.root != null)
                {
                   root!.Focus();
                }
                else if (App.Current.ApplicationLifetime is IClassicDesktopStyleApplicationLifetime desktop)
                {
                   desktop.MainWindow.Focus();
                }

                e.Handled = true;

            }
        }
    }

VirtualKeyboard

public enum VirtualKeyboardState
    {
        Default,
        Shift,
        Capslock,
        AltCtrl
    }
    public partial class VirtualKeyboard : UserControl
    {
        private static List<Type> Layouts { get; } = new List<Type>();
        private static Func<Type> DefaultLayout { get; set; }

        public static void AddLayout<TLayout>() where TLayout : KeyboardLayout => Layouts.Add(typeof(TLayout));

        public static void SetDefaultLayout(Func<Type> getDefaultLayout) => DefaultLayout = getDefaultLayout;

        public static async Task<string?> ShowDialog(TextInputOptions options, Window? owner = null)
        {
            var keyboard = new VirtualKeyboard();

            var window = new CoporateWindow();
            window.CoporateContent = keyboard;
            window.Title = "Keyboard";

            var mw = ((IClassicDesktopStyleApplicationLifetime)App.Current.ApplicationLifetime).MainWindow;

            await window.ShowDialog(owner ?? mw);
            if (window.Tag is string s)
            {
                //if (options.Source is TextBox tb)
                //    tb.Text = s;
                return s;
            }
            return null;
        }

        public TextBox TextBox_ { get; }
        public Button AcceptButton_ { get; }
        public string targetLayout { get; set; }
        public TransitioningContentControl TransitioningContentControl_ { get; }

        public IObservable<VirtualKeyboardState> KeyboardStateStream => _keyboardStateStream;
        private readonly BehaviorSubject<VirtualKeyboardState> _keyboardStateStream;

        private Window _parentWindow;

        public VirtualKeyboard()
        {
            InitializeComponent();
            TextBox_ = this.Get<TextBox>("TextBox");
            TransitioningContentControl_ = this.Get<Avalonia.Controls.TransitioningContentControl>("TransitioningContentControl");
            AcceptButton_ = this.Get<Button>("AcceptButton");

            AcceptButton_.AddHandler(Button.ClickEvent, acceptClicked);

            Initialized += async (sender, args) =>
            {

                if(targetLayout == null)
                {
                    TransitioningContentControl_.Content = Activator.CreateInstance(DefaultLayout.Invoke());
                }
                else
                {
                    var layout = Layouts.FirstOrDefault(x => x.Name.ToLower().Contains(targetLayout.ToLower()));
                    if (layout != null)
                    {
                        TransitioningContentControl_.Content = Activator.CreateInstance(layout);
                    }
                    else
                    {
                        TransitioningContentControl_.Content = Activator.CreateInstance(DefaultLayout.Invoke());
                    }
                }

                _parentWindow = this.GetVisualAncestors().OfType<Window>().First();
                await Task.Delay(TimeSpan.FromMilliseconds(100));
                Dispatcher.UIThread.Post(() =>
                {
                    TextBox_.Focus();
                    if (!string.IsNullOrEmpty(TextBox_.Text))
                        TextBox_.CaretIndex = TextBox_.Text.Length;
                });
            };

            KeyDown += (sender, args) =>
            {
                TextBox_.Focus();
                if (args.Key == Key.Escape)
                {
                    TextBox_.Text = "";
                }
                else if (args.Key == Key.Enter)
                {
                    _parentWindow.Tag = TextBox_.Text;
                    _parentWindow.Close();
                }
            };
            _keyboardStateStream = new BehaviorSubject<VirtualKeyboardState>(VirtualKeyboardState.Default);
        }

        private void acceptClicked(object? sender, RoutedEventArgs e)
        {
            _parentWindow.Tag = TextBox_.Text;
            _parentWindow.Close();
        }

        public void ProcessText(string text)
        {
            TextBox_.Focus();
            TextBox_.Text += text;
            //InputManager.Instance.ProcessInput(new RawTextInputEventArgs(KeyboardDevice.Instance, (ulong)DateTime.Now.Ticks, (Window)TextBox.GetVisualRoot(), text));
            if (_keyboardStateStream.Value == VirtualKeyboardState.Shift)
            {
                _keyboardStateStream.OnNext(VirtualKeyboardState.Default);
            }
        }

        public void Accept()
        {
            _parentWindow.Tag = TextBox_.Text;
            _parentWindow.Close();
        }

        public void ProcessKey(Key key)
        {
            if (key == Key.LeftShift || key == Key.RightShift)
            {
                if (_keyboardStateStream.Value == VirtualKeyboardState.Shift)
                {
                    _keyboardStateStream.OnNext(VirtualKeyboardState.Default);
                }
                else
                {
                    _keyboardStateStream.OnNext(VirtualKeyboardState.Shift);
                }
            }
            else if (key == Key.RightAlt)
            {
                if (_keyboardStateStream.Value == VirtualKeyboardState.AltCtrl)
                {
                    _keyboardStateStream.OnNext(VirtualKeyboardState.Default);
                }
                else
                {
                    _keyboardStateStream.OnNext(VirtualKeyboardState.AltCtrl);
                }
            }
            else if (key == Key.CapsLock)
            {
                if (_keyboardStateStream.Value == VirtualKeyboardState.Capslock)
                {
                    _keyboardStateStream.OnNext(VirtualKeyboardState.Default);
                }
                else
                {
                    _keyboardStateStream.OnNext(VirtualKeyboardState.Capslock);
                }
            }
            else
            {
                if (key == Key.Clear)
                {
                    TextBox_.Text = "";
                    TextBox_.Focus();
                }
                else if (key == Key.Enter || key == Key.ImeAccept)
                {
                    _parentWindow.Tag = TextBox_.Text;
                    _parentWindow.Close();
                }
                else if (key == Key.Help)
                {
                    _keyboardStateStream.OnNext(VirtualKeyboardState.Default);
                    if (TransitioningContentControl_.Content is KeyboardLayout layout)
                    {
                        var index = Layouts.IndexOf(layout.GetType());
                        if (Layouts.Count - 1 > index)
                        {
                            TransitioningContentControl_.Content = Activator.CreateInstance(Layouts[index + 1]);
                        }
                        else
                        {
                            TransitioningContentControl_.Content = Activator.CreateInstance(Layouts[0]);
                        }
                    }
                }
                else if(key == Key.Back)
                {

                    if(TextBox_.Text != null && TextBox_.Text.Length > 0)
                    {
                        TextBox_.Text = TextBox_.Text.Remove(TextBox_.Text.Length - 1, 1);
                    }

                }
                else
                {
                    TextBox_.Focus();

                    //InputManager.Instance.ProcessInput(new RawKeyEventArgs(KeyboardDevice.Instance, (ulong)DateTime.Now.Ticks, (Window)TextBox.GetVisualRoot(), RawKeyEventType.KeyDown, key, RawInputModifiers.None));
                    //InputManager.Instance.ProcessInput(new RawKeyEventArgs(KeyboardDevice.Instance, (ulong)DateTime.Now.Ticks, (Window)TextBox.GetVisualRoot(), RawKeyEventType.KeyUp, key, RawInputModifiers.None));
                }
            }
        }

        private void InitializeComponent()
        {
            AvaloniaXamlLoader.Load(this);
        }
    }

[011010101001]

Hello @nullx1337 thanks so much, i was searching for this for long, could you please share a working code that can compile with avalonia 11 ? thanks

[dariobattistella]

Hello @nullx1337 thanks so much, I tried to fix the keyboard for Avalonia 11 but i still have soma issues. Could you please share a working code that can compile with avalonia 11? thanks

[mozesa]

What about native On-Screen Keyboard support on Windows? i.e. when a TextBox gets focused, OSK should appear, just like in Android.

[dariobattistella]

Sure, but the app runs on embed system with linux-arm64, so, in this case i don't have any OSK.

[mozesa]

Do you need full keyboard or numerical is enough?

[dariobattistella]

full keyboard is mandatory

[krobotics]

I spent more than a week trying to get this working on an embedded lunix-arm64 system using @nullx1337 GotFocusEvent suggestion. There is a known bug in Avlonia-ui 11 that has to do with focus and popup dialogs. Every control you click after dismissing the keyboard with trigger the OnFocus event as the TextBox. To work around this I created a keyboard on the main window, instead of using a popup, and set it to visible on receiving a GotFocus event from a textbox. I also set the opacity of the other Grid item to 0.2.:

<Grid>
  <Grid x:Name="MainGrid" RowDefinitions="1*,10*,1*" ColumnDefinitions="1*,1*,1*,1*,1*" Opacity="{Binding MainOpacity}"> 
    <!-- Content -->
  </Grid>

  <Border IsVisible="{Binding IsOskVisible}"  Background="#40000000">
    <StackPanel HorizontalAlignment="Center" VerticalAlignment="Center">
      <cont:VirtualKeyboard x:Name="OSK" Background="LightGray"/>
    </StackPanel>
  </Border>
</Grid>

I use the Community toolkit Mvvm Messenger service to send 'show/hide' messages, to the MainWindow and VirtualKeyboard control and also to pass the source textbox object.

public class OskControlMsg : ValueChangedMessage<bool>
{
  public OskControlMsg(bool value) : base(value)
  {
  }
}

public class MainPageViewModel : ReactiveObject,  IRecipient<OskControlMsg>
{
...
  public void Receive(OskControlMsg message)
  {
    Dispatcher.UIThread.InvokeAsync(() =>
    {
      IsOskVisible = message.Value;
      MainOpacity = (IsOskVisible ? .2 : 1.0);
    });
  }
...
}

public partial class MainPage : ReactiveWindow<MainPageViewModel>
{
  public MainPage()
  {
...
      this.AddHandler<GotFocusEventArgs>(Control.GotFocusEvent, openVirtualKeyboard);
...
  }

  private void openVirtualKeyboard(object? sender, GotFocusEventArgs e)
  {
    if (e.Source.GetType() == typeof(TextBox))
    {
      if(!ViewModel.IsOskVisible)
      {
        WeakReferenceMessenger.Default.Send(new PassObjectMsg(e.Source));
        WeakReferenceMessenger.Default.Send(new OskControlMsg(true));
        e.Handled = true;
      }
      else
      {
        e.Handled = false;
      }
    }
  }
}

public class PassObjectMsg : ValueChangedMessage<object>
{
  public PassObjectMsg(object value) : base(value)
  {
  }
}

public partial class VirtualKeyboard : UserControl, IRecipient<PassObjectMsg>
{
 ...
  public async void Receive(PassObjectMsg message)
  {
    if (message.Value is TextBox textBox)
    {
      TextBox_.Text = textBox.Text;
      sourceObject = textBox;
      await Task.Delay(TimeSpan.FromMilliseconds(100));
      Dispatcher.UIThread.Post(() =>
      {
        TextBox_.Focus();
        if (!string.IsNullOrEmpty(TextBox_.Text))
          TextBox_.CaretIndex = TextBox_.Text.Length;
      });
    }
  }

  public void ProcessKey(Key key)
  {
...
      else if(key == Key.Escape)
      {
        WeakReferenceMessenger.Default.Send(new OskControlMsg(false));
      }
      else if (key == Key.Enter)
      {
        sourceObject.Text = TextBox_.Text;
        WeakReferenceMessenger.Default.Send(new OskControlMsg(false));
      }
'''
  }

}

As you don't need any of the popup stuff or ShowDialog you don't need to reference the window handle anywhere or have to find links back to some other objects. My application is reasonably simple, I insert Panels into the Main Grid instead of opening new windows so the keyboard will always be present. Hopefully mentioning this approach will help someone else with their project.

[c4801725870]

@vs-savelich I'm sorry, but I no longer use avalonia. 😐

what do you use now? I am starting out and would like to try other approaches. I am trying to find the fastest way to develop a modern looking UI on Raspberry Pi - I have used Visual Studio for Winforms and WPF for many years, hoping to find a cross platform solution.

[federicocodo]

@nullx1337 @krobotics Can you share the keyboard code please? I would need it for a project in avalonia, I also have a windows touch screen so I can check if it works and maybe help with the code