Avalonia sometimes discards the DispatcherOperation with Input or lower priority

[walterlv]

Describe the bug

If we Dispatcher.InvokeAsync an operation with Input or lower priority, it may be discarded by Avalonia.

To Reproduce

Let's see an example here:

1. Create an empty Avalonia app with an empty window. Add a PointerMoved event and slow it using Thread.Sleep(100).

public MainWindow()
{
    InitializeComponent();
    PointerMoved += OnPointerMoved;
}

private void OnPointerMoved(object? sender, PointerEventArgs e)
{
    // The grid has 200x200 size.
    var TestGrid = this.GetControl<Grid>("TestGrid");
    // Slow the event and reduce the fps.
    Thread.Sleep(100);
    // Change the RenderTransform which reprduce the bug mentioned here.
    TestGrid.RenderTransform = new TranslateTransform(e.GetPosition(this).X - 100, e.GetPosition(this).Y - 100);
    // Change the Margin which works fine.
    // TestGrid.Margin = new Thickness(e.GetPosition(this).X - 100, e.GetPosition(this).Y - 100);
}

2. Run the app and move the mouse.

If we change the RenderTransform property of a control in the PointerMoved event, the control will move 10fps at first but then it freezes. The PointerMoved event is still triggered, but the control doesn't move anymore.

If we change the Margin property of a control in the PointerMoved event, the control will always move 10fps.

Expected behavior

All operations should be executed even if they have Input or lower priority.

Avalonia version

11.1.3

OS

Windows

Additional context

No response

[walterlv]

Let's see deeper into the Avalonia source code.

1. The PointerMoved event stack trace.

WindowImpl.AppWndProc.cs file, line 818.

protected virtual unsafe IntPtr AppWndProc(IntPtr hWnd, uint msg, IntPtr wParam, IntPtr lParam)
{
    // ...
    if (e != null && Input != null)
    {
        Input(e);
    }
    // ...
}

It calls into the TopLevel.cs file, line 831.

private void HandleInput(RawInputEventArgs e)
{
    // ...
    Dispatcher.UIThread.Send(static state =>
    {
        // ...
        topLevel._inputManager?.ProcessInput(e);
    }, (this, e));
    // ...
}

It finally calls into the PointerMoved event.

All the calls are executed without any "priority" which means any input message comes the event will be triggered directly.

2. The "Signaled" stack trace.

The Dispatcher create its own message only window named AvaloniaMessageWindow which has its own WndProc method.

Win32Platform.cs file, line 141.

private IntPtr WndProc(IntPtr hWnd, uint msg, IntPtr wParam, IntPtr lParam)
{
    if (msg == (int)WindowsMessage.WM_DISPATCH_WORK_ITEM 
        && wParam.ToInt64() == Win32DispatcherImpl.SignalW 
        && lParam.ToInt64() == Win32DispatcherImpl.SignalL) 
        _dispatcher?.DispatchWorkItem();
    // ...
}

Every time the Dispatcher.InvokeAsync is called, it will send a message to the AvaloniaMessageWindow to dispatch the operation.

Dispatcher.Queue.cs file, line 192.

void ExecuteJobsCore(bool fromExplicitBackgroundProcessingCallback)
{
    // ...
    // We don't stop for executing jobs queued with >Input priority
    if (job.Priority > DispatcherPriority.Input)
    {
        ExecuteJob(job);
    }
    // If platform supports pending input query, ask the platform if we can continue running low priority jobs
    else if (_pendingInputImpl?.CanQueryPendingInput == true)
    {
        if (!_pendingInputImpl.HasPendingInput)
            ExecuteJob(job);
        else
        {
            RequestBackgroundProcessing();
            return;
        }
    }
    // ...
}

Notice that the priority larger than Input will be ExecuteJob but others will be ExecuteJob or RequestBackgroundProcessing with condition of HasPendingInput.

In this example, we slow the PointerMoved event using Thread.Sleep(100) which makes the HasPendingInput to be true so the operations later are all be sent to the RequestBackgroundProcessing method.

void RequestBackgroundProcessing()
{
    lock (InstanceLock)
    {
        if (_backgroundProcessingImpl != null)
        {
            if(_explicitBackgroundProcessingRequested)
                return;
            _explicitBackgroundProcessingRequested = true;
            _backgroundProcessingImpl.RequestBackgroundProcessing();
        }
        else if (_dueTimeForBackgroundProcessing == null)
        {
            _dueTimeForBackgroundProcessing = Now + 1;
            UpdateOSTimer();
        }
    }
}

Actually, it do nothing. So the operations are discarded.

The "Render" stack trace.

When we change the RenderTransform property of a control, it ScheduleRender. However, it use the Render priority at first but later it use the Input priority because of the reason mentioned in the comment below.

private void ScheduleRender(bool now)
{
    // Already scheduled, nothing to do
    if (_nextRenderOp != null)
    {
        if (now)
            _nextRenderOp.Priority = DispatcherPriority.Render;
        return;
    }
    // Sometimes our animation, layout and render passes might be taking more than a frame to complete
    // which can cause a "freeze"-like state when UI is being updated, but input is never being processed
    // So here we inject an operation with Input priority to check if Input wasn't being processed
    // for a long time. If that's the case the next rendering operation will be scheduled to happen after all pending input

    var priority = DispatcherPriority.Render;

    if (_inputMarkerOp == null)
    {
        _inputMarkerOp = _dispatcher.InvokeAsync(_inputMarkerHandler, DispatcherPriority.Input);
        _inputMarkerAddedAt = _time.Elapsed;
    }
    else if (!now && (_time.Elapsed - _inputMarkerAddedAt).TotalSeconds > MaxSecondsWithoutInput)
    {
        priority = DispatcherPriority.Input;
    }

    var renderOp = new DispatcherOperation(_dispatcher, priority, _render, throwOnUiThread: true);
    _nextRenderOp = renderOp;
    _dispatcher.InvokeAsyncImpl(renderOp, CancellationToken.None);
}

As a result, the real render will not happed because the RequestBackgroundProcessing discards it. The new ScheduleRender will not add because the _queuedUpdate is true which will only be set to false in the final Render operation in CompositingRenderer.UpdateCore method.

CompositingRenderer.cs file, line 80.

private void QueueUpdate()
{
    if(_queuedUpdate)
        return;
    _queuedUpdate = true;
    _compositor.RequestCompositionUpdate(_update);
}

CompositingRenderer.cs file, line 149.

private void UpdateCore()
{
    _queuedUpdate = false;
    // ...
}

---

In conclusion:

1. When change the RenderTransform property of a control, it queues Render priority operations at first but later it queues Input priority operations.
2. The RequestBackgroundProcessing method discards the Input priority operations.
3. The control freezes.

[lindexi]

Yeah, I can repro you issues. I just add the Rectangle to MainView.axaml and modify the TranslateTransform in PointerMoved.

The MainView.axaml:

  <Rectangle x:Name="Rectangle" Width="100" Height="100" Fill="Blue" HorizontalAlignment="Left" VerticalAlignment="Top">
      <Rectangle.RenderTransform>
          <TranslateTransform></TranslateTransform>
      </Rectangle.RenderTransform>
  </Rectangle>

The MainView.axaml.cs:

    public MainView()
    {
        InitializeComponent();

        PointerMoved += MainView_PointerMoved;
    }

    private void MainView_PointerMoved(object? sender, Avalonia.Input.PointerEventArgs e)
    {
        Thread.Sleep(100);
        var point = e.GetCurrentPoint(this);
        var translateTransform = (TranslateTransform) Rectangle.RenderTransform!;
        translateTransform.X = point.Position.X;
        translateTransform.Y = point.Position.Y;
    }

Run the application that you can find the application stop move the Rectangle when you move the mouse fast.

You can find my demo in github

[kekekeks]

Not executing Input-priority jobs when there is a pending user input matches XPF behavior:

https://github.com/dotnet/wpf/blob/30739d9d250860707da5864fbd79878c89ad027f/src/Microsoft.DotNet.Wpf/src/WindowsBase/System/Windows/Threading/Dispatcher.cs#L2901-L2902

https://github.com/dotnet/wpf/blob/30739d9d250860707da5864fbd79878c89ad027f/src/Microsoft.DotNet.Wpf/src/WindowsBase/System/Windows/Threading/Dispatcher.cs#L2035-L2047

Does dispatcher resume processing jobs eventually or is it stuck completely?

[walterlv]

@kekekeks Thanks for your source reference. I've read those codes and find that there is a difference between them:

• Avalonia will change the render priority to input after 1 second thich is set in DispatcherOptions while XPF do not.

After my test, I think the render operation should always use the Render priority without changing it to input. Without chaning the priority, the render is as fast as possible even though the UI is a little bit slow. Maybe I need to know more about the real reason why this happens. @kekekeks

BTW, setting the DispatcherOptions.InputStarvationTimeout value to a long time may improve the render performance and this can be a temporary solution.

[kekekeks]

Avalonia will change the render priority to input after 1 second thich is set in DispatcherOptions while XPF do not.

In WPF the input starvation interval is hardcoded to 500ms

Maybe I need to know more about the real reason why this happens.

It's needed for cases when input can't be processed at all because the animation/layout/render passes are consuming all available time, which could lead to an app being completely unresponsive while still drawing an animated progress bar or smth.

Does dispatcher resume processing jobs eventually or is it stuck completely?