Using keyboard and mouse on Linux with fbdev

[aguahombre]

Is it possible to use mouse and keyboard input with Linux framebuffer fbdev?

I have a working app using fbdev and touch on a Raspberry PI 3A running on Raspbian but no keyboard or mouse input. The same app is working on the Raspberry PI desktop with touch, keyboard and mouse.

[kekekeks]

Planned, but not implemented yet

[viniciusverasdossantos]

I also need. Congratulations on the project!

[dinonovak]

any update if support has been added or is in plan?

[dinonovak]

@aguahombre Can you please advise how did you manage to enable touchscreen under framebuffer? I have 5inch hdmi display with xpt2046 touch controller, but I do not know how to enable touch

[aguahombre]

@dinonovak Last time I looked, the Linux framebuffer was not well supported and I was advised by kekekeks (I think) to run my app in kiosk mode under X. I found this works quite well.

[Drise13]

I'm looking to get keyboard events into the Avalonia ControlCatalog sample for a linux framebuffer device (BeagleBoneBlack). I have touch inputs working (through libts i think) and output to the framebuffer using --fbdev. I can't seem to get usb keyboard events into the application however, even though it looks like I have input focus captured on a textbox.

Is there any updates to capturing keyboard events while running a framebuffer output?

[masterworgen]

Planned, but not implemented yet

It's been 3 years, where is the keyboard?

[kekekeks]

Still planned. And will stay planned if people will be expressing that kind of attitude, BTW.

[Oaz]

In case anyone landing here is interested, I've got an experimental extended LibInputBackend with keyboard handling. It works for basic use cases but is clearly not production ready.

[sn4k3]

In case anyone landing here is interested, I've got an experimental extended LibInputBackend with keyboard handling. It works for basic use cases but is clearly not production ready.

I landed here and have to say that this works perfectly :) Thanks! Would be nice to see it as nuget package ;)

Note: Tab for focus scroll works fine, however SHIFT + TAB does nothing. Also accent letters that require accent key + letter eg: (´e to make: é, as well ~a to make ã) doesn't work.

I also did this changes to allow new devices to be listen, eg: bluetooth or new usb:

// On InputThread:
foreach (var f in options.Events!)
{
    libinput_path_add_device(ctx, f);
    options.ActiveEvents.Add(f);
}

while (true)
{

    if (options.ListenEventsChanges && (DateTime.Now - lastUpdate).TotalMilliseconds >= updateInterval)
    {
        var events = Directory.GetFiles("/dev/input", "event*");
        for (var i = options.ActiveEvents.Count - 1; i >= 0; i--)
        {
            var f = options.ActiveEvents[i];
            if (!events.Contains(f))
            {
                options.ActiveEvents.RemoveAt(i);
            }
        }
        foreach (var f in events)
        {
            if (options.ActiveEvents.Contains(f)) continue;
            options.ActiveEvents.Add(f);
            libinput_path_add_device(ctx, f);
        }
        lastUpdate = DateTime.Now;
    }

[MasterMann]

I also did this changes to allow new devices to be listen, eg: bluetooth or new usb:

@sn4k3 can you post your LibInputBackendOptions code? Seems like some of the changes were done there too, but missing from your snippet

[sn4k3]

Yes, you need to add missing parts to config. Meanwhile I made more changes, here the complete files to replace:

LibInputBackendOptions.cs

#nullable enable
using System.Collections.Generic;

namespace Avalonia.LibInputExperiments;

/// <summary>
/// LibInputBackend Options.
/// </summary>
public sealed record class LibInputBackendOptions
{
    /// <summary>
    /// Sets to listen for event changes, if true it will listen for events changes and register them to make the device work without restarting the application.
    /// </summary>
    public bool ListenEventsChanges { get; init; }

    /// <summary>
    /// Sets the listen for event changes interval in milliseconds.<br/>
    /// Requires <see cref="ListenEventsChanges"/> to be true.
    /// </summary>
    public int ListenEventsChangesInterval { get; init; } = 5000;

    /// <summary>
    /// List Events of events handler to monitoring eg: /dev/eventX.
    /// </summary>
    public IReadOnlyList<string>? Events { get; init; } = null;
    public LibInputKeyboardConfiguration Keyboard { get; init; } = new ();
}

public sealed record class LibInputKeyboardConfiguration
{
    public string Rules { get; init; } = string.Empty;
    public string Model { get; init; } = string.Empty;
    public string Layout { get; init; } = string.Empty;
    public string Variant { get; init; } = string.Empty;
    public string Options { get; init; } = string.Empty;
}

LibInputBackend.cs

#nullable enable
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading;
using Avalonia.Input;
using Avalonia.Input.Raw;
using Avalonia.LibInputExperiments.Helpers;
using Avalonia.LinuxFramebuffer.Input;
using static Avalonia.LibInputExperiments.LibInputNativeUnsafeMethods;

namespace Avalonia.LibInputExperiments
{
    public partial class LibInputBackend : IInputBackend
    {
        private IScreenInfoProvider? _screen;
        private IInputRoot? _inputRoot;
        private const string LibInput = nameof(LinuxFramebuffer) + "/" + nameof(Input) + "/" + nameof(LibInput);
        private Action<RawInputEventArgs>? _onInput;
        private readonly LibInputBackendOptions? _options;

        public LibInputBackend(LibInputBackendOptions? options = default)
        {
            _options = options;
        }

        private string[] GetAvailableEvents()
        {
            return Directory.GetFiles("/dev/input", "event*");
        }

        private unsafe void InputThread(IntPtr ctx, LibInputBackendOptions options)
        {
            SetupKeyboard(options.Keyboard);
            var fd = libinput_get_fd(ctx);

            var activeEvents = new Dictionary<string, IntPtr>();

            foreach (var f in options.Events!)
            {
                var instance = libinput_path_add_device(ctx, f);
                activeEvents.TryAdd(f, instance);
            }

            var lastPoolUpdate = DateTime.Now;

            while (true)
            {
                if (options.ListenEventsChanges && (DateTime.Now - lastPoolUpdate).TotalMilliseconds >= options.ListenEventsChangesInterval)
                {
                    var events = GetAvailableEvents();

                    // Checks if the get events and active events both contain same events
                    foreach (var f in activeEvents.Keys)
                    {
                        if (events.Contains(f)) continue;
                        if (activeEvents.Remove(f, out var inputPtr))
                        {
                            // This create segmentation fault, not sure how to handle this
                            // For now we do not remove old devices ptr
                            /*if (inputPtr != IntPtr.Zero)
                            {
                                libinput_path_remove_device(inputPtr);
                            }*/
                        }
                    }

                    // Adds events if not present in active events
                    foreach (var f in events)
                    {
                        if (activeEvents.ContainsKey(f)) continue;
                        var inputPtr = libinput_path_add_device(ctx, f);
                        activeEvents.TryAdd(f, inputPtr);
                    }
                    lastPoolUpdate = DateTime.Now;
                }

                IntPtr ev;
                libinput_dispatch(ctx);
                while ((ev = libinput_get_event(ctx)) != IntPtr.Zero)
                {
                    var type = libinput_event_get_type(ev);

                    if (type >= LibInputEventType.LIBINPUT_EVENT_KEYBOARD_KEY
                        && type <= LibInputEventType.LIBINPUT_EVENT_KEYBOARD_KEY)
                        HandleKeyboard(ev, type);

                    if (type >= LibInputEventType.LIBINPUT_EVENT_TOUCH_DOWN &&
                        type <= LibInputEventType.LIBINPUT_EVENT_TOUCH_CANCEL)
                        HandleTouch(ev, type);

                    if (type >= LibInputEventType.LIBINPUT_EVENT_POINTER_MOTION
                        && type <= LibInputEventType.LIBINPUT_EVENT_POINTER_AXIS)
                        HandlePointer(ev, type);

                    // if (type >= LibInputEventType.LIBINPUT_EVENT_TABLET_TOOL_AXIS
                    //     && type <= LibInputEventType.LIBINPUT_EVENT_TABLET_TOOL_BUTTON)
                    //     HandleTabletTool(ev, type);
                    //
                    // if (type >= LibInputEventType.LIBINPUT_EVENT_TABLET_PAD_BUTTON
                    //     && type <= LibInputEventType.LIBINPUT_EVENT_TABLET_PAD_STRIP)
                    //     HandleTabletPad(ev, type);
                    //
                    // if (type >= LibInputEventType.LIBINPUT_EVENT_GESTURE_SWIPE_BEGIN
                    //     && type <= LibInputEventType.LIBINPUT_EVENT_GESTURE_PINCH_END)
                    //     HandleGesture(ev, type);

                    libinput_event_destroy(ev);
                    libinput_dispatch(ctx);
                }

                pollfd pfd = new pollfd { fd = fd, events = 1 };
                NativeUnsafeMethods.poll(&pfd, new IntPtr(1), 10);
            }
        }

        private void ScheduleInput(RawInputEventArgs ev) => _onInput?.Invoke(ev);

        public void Initialize(IScreenInfoProvider screen, Action<RawInputEventArgs> onInput)
        {
            _screen = screen;
            _onInput = onInput;
            var ctx = libinput_path_create_context();
            var options = new LibInputBackendOptions()
            {
                ListenEventsChanges = _options?.ListenEventsChanges ?? false,
                ListenEventsChangesInterval = _options?.ListenEventsChangesInterval ?? 5000,
                Events = _options?.Events ?? GetAvailableEvents(),
                Keyboard = _options?.Keyboard ?? new LibInputKeyboardConfiguration()
            };

            new Thread(() => InputThread(ctx, options))
            {
                Name = "Input Manager Worker",
                IsBackground = true
            }.Start();
        }

        public void SetInputRoot(IInputRoot root)
        {
            _inputRoot = root;
        }
    }
}

Note: I wanted to libinput_path_remove_device(inputPtr); when a device is removed, but I got segmentation fault, so I skip that ideia and register again. It works well for BT, if you disconnect / connect a keyboard it will always work.

[Oaz]

Yes, you need to add missing parts to config. Meanwhile I made more changes, here the complete files to replace:

LibInputBackendOptions.cs

#nullable enable
using System.Collections.Generic;

namespace Avalonia.LibInputExperiments;

@sn4k3 if you want changes in this LibInputExperiments project, they shall not be posted here in an Avalonia UI issue. Please open an issue, or, better, a pull request in the LibInputExperiments project (it is not related to official Avalonia UI projects).