Debug.Fail(...) crashes the Out-Of-Process WinForms Designer

SteffenSchwaiger commented 5 months ago

Environment

17.10.0 Preview 2.0

.NET version

.NET 8

Did this work in a previous version of Visual Studio and/or previous .NET release?

It worked in .NET Framework / In-Process Designer.

Issue description

When using a common form base class for multiple other form classes, it is not yet possible to mark that class as abstract, or the designer won't load. The team seems to be aware of this and is tracking this feature under the following issue: https://github.com/dotnet/winforms-designer/issues/3142.

We work around this limitation of the WinForms Designer (In-Process & Out-Of-Process) by not marking the base class as abstract and calling Debug.Fail(...) in each of the "abstract" methods to "ensure" that the class is not directly used by developers:

In the old designer, this meant that an assertion was displayed when the design view was opened, but after that the designer could be used as normal.

In the new designer, dialog instantiation fails and the user interface gets stuck on the current screen:

Steps to reproduce

Create two UserControls:

public partial class CustomUserControlBase : UserControl
{
    public CustomUserControlBase()
    {
        InitializeComponent();

        RunCode();
    }

    protected virtual void RunCode()
    {
        Debug.Fail("this method has to be implemented from the derived class");
    }
}

public partial class CustomUserControlDerived : CustomUserControlBase
{
    public CustomUserControlDerived()
    {
        InitializeComponent();
    }
}

Open the designer for the derived user control.

Diagnostics

[10:58:50.295483] fail: [Demo.Controls]: Process terminated. Assertion failed.
[10:58:50.295483] fail: [Demo.Controls]: this method has to be implemented from the derived class
[10:58:50.295483] fail: [Demo.Controls]:    at Demo.Controls.DisplayUserControlBase.ResizeControl()
[10:58:50.295483] fail: [Demo.Controls]:    at Demo.Controls.DisplayUserControlBase.OnResize(EventArgs e)
[10:58:50.295483] fail: [Demo.Controls]:    at System.Windows.Forms.Control.OnSizeChanged(EventArgs e)
[10:58:50.295483] fail: [Demo.Controls]:    at System.Windows.Forms.Control.UpdateBounds(Int32 x, Int32 y, Int32 width, Int32 height, Int32 clientWidth, Int32 clientHeight)
[10:58:50.295483] fail: [Demo.Controls]:    at System.Windows.Forms.Control.UpdateBounds()
[10:58:50.295483] fail: [Demo.Controls]:    at System.Windows.Forms.Control.WmWindowPosChanged(Message& m)
[10:58:50.295483] fail: [Demo.Controls]:    at System.Windows.Forms.Control.WndProc(Message& m)
[10:58:50.295483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Designers.ControlDesigner.DesignerWindowTarget.DefWndProc(Message& m)
[10:58:50.295483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Designers.ControlDesigner.DefWndProc(Message& m)
[10:58:50.295483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Designers.ControlDesigner.WndProc(Message& m)
[10:58:50.295483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Designers.ScrollableControlDesigner.WndProc(Message& m)
[10:58:50.295483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Designers.DocumentDesigner.WndProc(Message& m)
[10:58:50.295483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Designers.ControlDesigner.DesignerWindowTarget.OnMessage(Message& m)
[10:58:50.295483] fail: [Demo.Controls]:    at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
[10:58:50.295483] fail: [Demo.Controls]:    at System.Windows.Forms.NativeWindow.Callback(HWND hWnd, MessageId msg, WPARAM wparam, LPARAM lparam)
[10:58:50.295483] fail: [Demo.Controls]:    at Windows.Win32.PInvoke.SetWindowPos(HWND hWnd, HWND hWndInsertAfter, Int32 X, Int32 Y, Int32 cx, Int32 cy, SET_WINDOW_POS_FLAGS uFlags)
[10:58:50.295483] fail: [Demo.Controls]:    at Windows.Win32.PInvoke.SetWindowPos[T1,T2](T1 hWnd, T2 hWndInsertAfter, Int32 X, Int32 Y, Int32 cx, Int32 cy, SET_WINDOW_POS_FLAGS uFlags)
[10:58:50.295483] fail: [Demo.Controls]:    at System.Windows.Forms.Control.SetBoundsCore(Int32 x, Int32 y, Int32 width, Int32 height, BoundsSpecified specified)
[10:58:50.295483] fail: [Demo.Controls]:    at System.Windows.Forms.Control.SetBounds(Int32 x, Int32 y, Int32 width, Int32 height, BoundsSpecified specified)
[10:58:50.295483] fail: [Demo.Controls]:    at System.Windows.Forms.Control.set_Size(Size value)
[10:58:50.295483] fail: [Demo.Controls]:    at System.Windows.Forms.Control.SetClientSizeCore(Int32 x, Int32 y)
[10:58:50.295483] fail: [Demo.Controls]:    at System.Windows.Forms.Control.set_ClientSize(Size value)
[10:58:50.295483] fail: [Demo.Controls]:    at Microsoft.WinForms.DesignTools.Designers.UserControlDocumentDesigner.set_Size(Size value)
[10:58:50.295483] fail: [Demo.Controls]:    at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
[10:58:50.295483] fail: [Demo.Controls]:    at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
[10:58:50.296483] fail: [Demo.Controls]:    at System.Reflection.MethodBaseInvoker.InvokeWithOneArg(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
[10:58:50.296483] fail: [Demo.Controls]:    at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
[10:58:50.296483] fail: [Demo.Controls]:    at System.ComponentModel.ReflectPropertyDescriptor.SetValue(Object component, Object value)
[10:58:50.296483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Serialization.CodeDomSerializerBase.DeserializePropertyAssignStatement(IDesignerSerializationManager manager, CodeAssignStatement statement, CodePropertyReferenceExpression propertyReferenceEx, Boolean reportError)
[10:58:50.296483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Serialization.CodeDomSerializerBase.DeserializeAssignStatement(IDesignerSerializationManager manager, CodeAssignStatement statement)
[10:58:50.296483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Serialization.CodeDomSerializerBase.DeserializeStatement(IDesignerSerializationManager manager, CodeStatement statement)
[10:58:50.296483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Serialization.CodeDomSerializer.DeserializeStatementToInstance(IDesignerSerializationManager manager, CodeStatement statement)
[10:58:50.296483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Serialization.CodeDomSerializer.Deserialize(IDesignerSerializationManager manager, Object codeObject)
[10:58:50.296483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Serialization.ControlCodeDomSerializer.Deserialize(IDesignerSerializationManager manager, Object codeObject)
[10:58:50.296483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Serialization.TypeCodeDomSerializer.DeserializeName(IDesignerSerializationManager manager, String name, CodeStatementCollection statements)
[10:58:50.296483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Serialization.TypeCodeDomSerializer.Deserialize(IDesignerSerializationManager manager, CodeTypeDeclaration declaration)
[10:58:50.296483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Sessions.Session.DeserializeToRootComponent(CodeTypeDeclaration typeDeclaration, String rootComponentClassName)
[10:58:50.296483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Sessions.Session.DeserializeRootComponent(CodeTypeDeclaration typeDeclaration, String rootComponentClassName, ResourceContentData[] resourceDocDataContent, String basePath)
[10:58:50.296483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Protocol.Endpoints.Sessions.InitializeRootComponentHandler.HandleRequest(InitializeRootComponentRequest request)
[10:58:50.296483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Protocol.Endpoints.RequestHandler`2.Microsoft.DotNet.DesignTools.Protocol.Endpoints.IRequestHandler.HandleRequest(Request request)
[10:58:50.296483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Protocol.Endpoints.RequestManager.HandleRequestAsync(String name, Request request)
[10:58:50.296483] fail: [Demo.Controls]:    at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object s)
[10:58:50.296483] fail: [Demo.Controls]:    at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
[10:58:50.296483] fail: [Demo.Controls]:    at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext(Thread threadPoolThread)
[10:58:50.296483] fail: [Demo.Controls]:    at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.MoveNext()
[10:58:50.296483] fail: [Demo.Controls]:    at System.Windows.Forms.Control.InvokeMarshaledCallbackHelper(Object obj)
[10:58:50.296483] fail: [Demo.Controls]:    at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
[10:58:50.296483] fail: [Demo.Controls]:    at System.Windows.Forms.Control.InvokeMarshaledCallbacks()
[10:58:50.296483] fail: [Demo.Controls]:    at System.Windows.Forms.Control.WndProc(Message& m)
[10:58:50.296483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Server.Window.ServerWindow.WndProc(Message& m)
[10:58:50.296483] fail: [Demo.Controls]:    at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
[10:58:50.296483] fail: [Demo.Controls]:    at System.Windows.Forms.NativeWindow.Callback(HWND hWnd, MessageId msg, WPARAM wparam, LPARAM lparam)
[10:58:50.296483] fail: [Demo.Controls]:    at Windows.Win32.PInvoke.DispatchMessage(MSG* lpMsg)
[10:58:50.296483] fail: [Demo.Controls]:    at System.Windows.Forms.Application.ComponentManager.Microsoft.Office.IMsoComponentManager.FPushMessageLoop(UIntPtr dwComponentID, msoloop uReason, Void* pvLoopData)
[10:58:50.296483] fail: [Demo.Controls]:    at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(msoloop reason, ApplicationContext context)
[10:58:50.297483] fail: [Demo.Controls]:    at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(msoloop reason, ApplicationContext context)
[10:58:50.297483] fail: [Demo.Controls]:    at Microsoft.DotNet.DesignTools.Server.DesignToolsServer.<>c__DisplayClass54_0.<StartUIThreadAsync>b__1()
[10:58:50.297483] fail: [Demo.Controls]:    at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
[10:58:54.014493] fail: StreamJsonRpc.ConnectionLostException: The JSON-RPC connection with the remote party was lost before the request could complete.
                           at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                           at StreamJsonRpc.JsonRpc.<InvokeCoreAsync>d__165.MoveNext()
                        --- End of stack trace from previous location where exception was thrown ---
                           at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                           at StreamJsonRpc.JsonRpc.<InvokeCoreAsync>d__154`1.MoveNext()
                        --- End of stack trace from previous location where exception was thrown ---
                           at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
                           at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
                           at Microsoft.DotNet.DesignTools.Client.DesignToolsClient.<SendRequestAsync>d__49`1.MoveNext()

                        For information on how to troubleshoot the designer refer to the guide at https://aka.ms/winforms/designer/troubleshooting.
[10:58:54.050493] info: VsDesignerLoader::PerformLoad::LoadCompletionTask - Calling DependentLoadComplete.

RussKie commented 5 months ago

As you said, Debug.Fail(...) crashes the out-of-process designer, and the VS client is unable to communicate with the crashed process... What are your expectations?

SteffenSchwaiger commented 5 months ago

I would expect one of the following:

The DesignToolsServer behaves similarly as before (in-process designer), i.e. an "Assertion failed..." dialog is displayed, which can be closed by the user.
Debug.Assert(...) or Debug.Fail(...) calls are caught inside the DesignToolsServer and are only logged to the "Windows Forms" output in Visual Studio. The Visual Designer itself works as if these assertions were not present.

RussKie commented 5 months ago

Thanks for sharing the ideas. The first won't work, as it'd block the remote process and, thus, block the VS, which is a big no-no. The second is likely achievable with something like NoAssertContext.

Though, what would you expect to happen on the designer surface?

SteffenSchwaiger commented 5 months ago

The first won't work, as it'd block the remote process and, thus, block the VS, which is a big no-no.

I can understand this. Is it not possible to inform Visual Studio via RPC that the designer is in an assertion state and Visual Studio could display the same assertion dialog as in the old designer?

The second is likely achievable with something like NoAssertContext.

Sounds promising to me.

Though, what would you expect to happen on the designer surface?

Since the assertion in our case is only relevant when the application is running, the designer surface could simply ignore it. In this case, the designer surface would at least not get stuck at the "Loading designer..." stage like it currently is.

We work around this problem by only calling Debug.Fail(...) when the control is not instanciated by the DesignToolsServer process. Sample pseudocode:

private static bool _designMode => Assembly.GetEntryAssembly()?.GetName()?.Name?.Contains("DesignToolsServer") ?? false;

protected virtual void ResizeControl()
{
    if (!_designMode)
        Debug.Fail("this method has to be implemented from the derived class");
}

Another more elegant way would be to display the assertion and let the user decide whether to still load the control/dialog/... or close the current designer tab.

Sample: VSDesignerAssertion

Shyam-Gupta commented 5 months ago

I think we can probably implement a custom TraceListener in server process which will route the Debug.Fail messages to output window or send them as notification for display in a MessageBox.

merriemcgaw commented 5 months ago

@Olina-Zhang can you copy this issue to the designer repo?

Syareel-Sukeri commented 5 months ago

@merriemcgaw Filed a DT issue: https://github.com/microsoft/winforms-designer/issues/5926 for this issue

Shyam-Gupta commented 3 months ago

Fix for this bug has been implemented and should be available in an upcoming VS 17.11 preview release. Thanks.

SteffenSchwaiger commented 2 months ago

A message box is shown in Visual Studio 17.11.0 Preview 4.0 which displays the assertion. After confirming the dialog with OK, the Designer is loaded. :tada:

dotnet / winforms