AvaloniaUI / Avalonia

Develop Desktop, Embedded, Mobile and WebAssembly apps with C# and XAML. The most popular .NET UI client technology
https://avaloniaui.net
MIT License
25.28k stars 2.19k forks source link

NullReferenceException during Rendering ( Avalonia.Rendering.SceneGraph.Scene.Clone ) #11113

Closed Tetedeiench closed 1 year ago

Tetedeiench commented 1 year ago

Describe the bug I have several reports of NullReferenceException occuring during the render loop. They seem out of my control at first glance and are purely render based. I am not sure this is fully related to Avalonia, due to a third party code, or even mine.

I couldn't reproduce the issue manually - the steps for it to occur are currently unknown.

It seems as if a Clone function returns null at some point, and then proceed to fail when adding the clone to a dictionary ( which is guaranteed not to be null at this point, reading the function correctly).

You may not be able to give me the exact reason as the way to reproduce it is currently unknown ( iwould actually be very imopressed if you do), but giving me directions on how to pinpoint the issue would be awesome ( what could trigger this, what I should look for, etc).

Again, I am not sure this is an avalonia bug, or a side-effect of something I did wrong !

To Reproduce Unknown at this point - I only have automated stack traces uploaded through my crash handler. Sorry :(

Expected behavior There should be no crash !

Screenshots putting a screenshot of the most complex app screen as it may give you a hint image

Desktop (please complete the following information):

Additional context The app is using heavily Oxyplot.Avalonia , DataGrid, Material.Avalonia, ReactiveUI/DynamicData, all in their latest versions.

Also, my app put the CPU / Memory under heavy use, so it makes race conditions VERY likely to appear ( it's a stress test )

Stack trace

System.NullReferenceException: Object reference not set to an instance of an object.
at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 166
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.CloneScene() in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 91
at Avalonia.Rendering.DeferredRenderer.UpdateScene() in /_/src/Avalonia.Visuals/Rendering/DeferredRenderer.cs:line 639
at Avalonia.Rendering.DeferredRenderer.UpdateSceneIfNeeded() in /_/src/Avalonia.Visuals/Rendering/DeferredRenderer.cs:line 623
at Avalonia.Threading.JobRunner.Job.Avalonia.Threading.JobRunner.IJob.Run() in /_/src/Avalonia.Base/Threading/JobRunner.cs:line 181
at Avalonia.Threading.JobRunner.RunJobs(Nullable`1 priority) in /_/src/Avalonia.Base/Threading/JobRunner.cs:line 37
at Avalonia.Win32.Win32Platform.WndProc(IntPtr hWnd, UInt32 msg, IntPtr wParam, IntPtr lParam) in /_/src/Windows/Avalonia.Win32/Win32Platform.cs:line 263
at Avalonia.Win32.Interop.UnmanagedMethods.DispatchMessage(MSG& lpmsg)
at Avalonia.Win32.Win32Platform.RunLoop(CancellationToken cancellationToken) in /_/src/Windows/Avalonia.Win32/Win32Platform.cs:line 210
at Avalonia.Threading.Dispatcher.MainLoop(CancellationToken cancellationToken) in /_/src/Avalonia.Base/Threading/Dispatcher.cs:line 65
at Avalonia.Controls.ApplicationLifetimes.ClassicDesktopStyleApplicationLifetime.Start(String[] args) in /_/src/Avalonia.Controls/ApplicationLifetimes/ClassicDesktopStyleApplicationLifetime.cs:line 120
at Avalonia.ClassicDesktopStyleApplicationLifetimeExtensions.StartWithClassicDesktopLifetime[T](T builder, String[] args, ShutdownMode shutdownMode) in /_/src/Avalonia.Controls/ApplicationLifetimes/ClassicDesktopStyleApplicationLifetime.cs:line 209
maxkatz6 commented 1 year ago

DeferredRenderer was completely removed from the framework in the 11.0 previews. This isn't an issue anymore, as bugged code does not exist anymore.

Tetedeiench commented 1 year ago

That's good news ! However, 11 isn't stable yet, so I can't exactly migrate from 10.0 to 11.0 in production right now, can I ?

Or is 11 stable enough to allow for such a step ?

timunie commented 1 year ago

Some use it in production already, but be carefully and make sure to read breaking changes wiki. Probably make a copy to try it out and see if it's already stable enough for your usecase.

Tetedeiench commented 1 year ago

Some use it in production already, but be carefully and make sure to read breaking changes wiki. Probably make a copy to try it out and see if it's already stable enough for your usecase.

Tried migrating - the scale of the upgrade is huge ( quite some breaking changes). Could go through them all fast just to get things compiled, but there are lots of issues with that upgrade.

The most blatant one is my plotting library isn't up to date, and while there is a version in an alpha feed having avalonia 11 support, it's clearly not suited for production.

I'm two weeks away from stable release, this is not really the time to change the major version of avalonia :(

Is Avalonia v10 unsupported right now ? I understand it is annoying to fix a bug in an older version when there's a shiny new one in the works.

Are we sure this is due to v10 code and deferredrenderer ?

Tetedeiench commented 1 year ago

Spent some time in the code and I can't wrap my head on why this could be occuring.

If anyone can give me pointers on how to find this issue, I'm all for it. Looking at the Scene.Clone() function, I don't see how this could be caused by avalonia's code, since there's basically no Dictionary involved .

My guess would be the clone() function somehow is overriden or calls external code from a component I'm using, which leads to this exception, but I have no clue how to find it :(

timunie commented 1 year ago

@Tetedeiench most activity goes into 11.0 atm, so it's unlikely to get a fast fix for 0.10 I guess. If you have paid support you can ask the team if there is a chance to solve it for you, but I cannot promise anything. However, if I were you, I'd probably clone the source of Avalonia (0.10.x-stable branch) and then debug it from your source. Avalonia would be added as project reference rather than nuget. You could also try out to reproduce in Sandbox.

Good luck 👍

Tetedeiench commented 1 year ago

Well, I don't have paid support ( much too small for that - 6.5k$ is way out of my league) and I'm not greedy enough to ask for a fast fix here.

Fixing it myself is fine as well, but I have to admit I don't really know where to start for fixing this ! Especially since it isn't 100% reproductible.

If anyone could give me pointers, maybe just a comment like what this could be caused by, it would help tremendously :)

Tetedeiench commented 1 year ago

Still working on it ( sorry for the tons of replies ).

Looking at another stack trace I just received, this is starting to get interesting. The new one I got :

System.ArgumentNullException: Value cannot be null.
at Avalonia.Rendering.SceneGraph.VisualNode..ctor(IVisual visual, IVisualNode parent) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/VisualNode.cs:line 35
at Avalonia.Rendering.SceneGraph.VisualNode.Clone(IVisualNode parent) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/VisualNode.cs:line 264
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 166
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181
at Avalonia.Rendering.SceneGraph.Scene.Clone(VisualNode source, IVisualNode parent, Dictionary`2 index) in /_/src/Avalonia.Visuals/Rendering/SceneGraph/Scene.cs:line 181

Analyzing the stack trace, we tried to clone a VisualNode by calling Clone, which does construct a new VisualNode. This is an extract from the class VisualNode.cs (doing my best with indentation here) :

    internal class VisualNode : IVisualNode
    {
    (...)
        /// <summary>
        /// Initializes a new instance of the <see cref="VisualNode"/> class.
        /// </summary>
        /// <param name="visual">The visual that this node represents.</param>
        /// <param name="parent">The parent scene graph node, if any.</param>
        public VisualNode(IVisual visual, IVisualNode parent)
        {
            Contract.Requires<ArgumentNullException>(visual != null);

            Visual = visual;
            Parent = parent;
            HasAncestorGeometryClip = parent != null && 
                (parent.HasAncestorGeometryClip || parent.GeometryClip != null);
        }
        /// <inheritdoc/>
        public IVisual Visual { get; }

(...)

        public VisualNode Clone(IVisualNode parent)
        {
            return new VisualNode(Visual, parent)
            {
                Transform = Transform,
                ClipBounds = ClipBounds,
                ClipToBoundsRadius =  ClipToBoundsRadius,
                ClipToBounds = ClipToBounds,
                LayoutBounds = LayoutBounds,
                GeometryClip = GeometryClip,
                _opacity = Opacity,
                OpacityMask = OpacityMask,
                _drawOperations = _drawOperations,
                _drawOperationsRefCounter = _drawOperationsRefCounter?.Clone(),
                _drawOperationsCloned = true,
                LayerRoot= LayerRoot,
            };
        }

The exception was caused by :

Contract.Requires<ArgumentNullException>(visual != null);

This is VERY interesting, as a VisualNode cannot be constructed without a Visual, and the Visual property is get only, so modifications outside the constructor is out of the equation.

Yet, we are trying to clone a VisualNode without a visual. How is that possible ?

I've seen this happen with Serialization, but it's obviously out the equation here, so my guess would be a GC occuring at the wrong time and borking things.

Thoughts ?

maxkatz6 commented 1 year ago

Just to be clear, it's very unlikely this issue will be fixed in the stable release. Whole SceneGraph doesn't exist anymore in 11.0.

Yet, we are trying to clone a VisualNode without a visual. How is that possible ?

Not sure, but if it's being executed on the render thread (I am not sure), there could be a race condition somewhere.

kekekeks commented 1 year ago

DeferredRenderer is removed

Tetedeiench commented 1 year ago

@maxkatz6 @kekekeks The code is present in the current stable branch at the time I'm writing this ( 0.10 is still the official release, right ?).

I'm fine with you doing so ( I have no paid sub, so I can't ask for anything really).

The question is this one then : Do you still support 0.10 and issue bugfixes ?

This makes it look like you're not supporting it anymore to be honest.

I must admit it's a tad concerning for me, as it implies I am forced to use a preview or RC branch to get "community level" support in the future.

If that's the case, please make it clear to everyone !

timunie commented 1 year ago

Critical bug fixes still go into 0.10.x. but it's harder atm as the master branch differs a lot now. All the work we do is to get a smoother experience later. I hope for your understanding.

Sidenote: 11.0 RC2 may be the last "preview" before stable release. 🤞

Tetedeiench commented 1 year ago

@timunie thanks for your answer, but I still don't know if 0.10 is still supported since the reason for closing this issue is "the code doesn't exist in the current preview".

This is a tad critical in building trust for using the framework - at least for me. And others who might stumble upon this issue.

Again, I am just a community user so I'm not in position to request anything. I'll see my app fail in prod until v11 is released and all libraries are ported to that branch which will take some time. As expected from someone without a subscription, which is fine again.

But this is... Surprising to say the least. And overall I'm concerned at the signal this sends to us users.

Gillibald commented 1 year ago

Avalonia 0.10. will only receive a few bugfixes. You need to work around it or move to Avalonia 11

timunie commented 1 year ago

You're welcome to our telegram chat if you need further support. Updating soon is probably a good idea anyway since the benefits are huge.

Tetedeiench commented 1 year ago

Well, @Gillibald said it, despite being the stable version, 0.10 is effectively abandoned :(

I can't switch to 0.11 since a critical library I use ( oxyplot ) doesn't have a 0.11 compatible version, just something in progress in their issue section. Also, the port effort is very big (already tried it - this is not an easy update by a long shot), and I can't do that overnight. That's a major release if anything.

Frankly, I would have expected 0.10 to get some support for a few months after 0.11 official release. Not for it to end before its successor's official release. I can't exactly use a preview branch in production, especially with all the ecosystem not up to date.

As I am a free user, again, this is fine, further demands would require me to pay for premium support, which I fully understand. It's disappointing from my point of view, but again, fully understandable.

Just be blunt with it, and make 0.11 the stable release, since 0.10 isn't getting support as of today (or, looking at this issue's first comment which is like "this doesn't exists in 0.11", as of April 24th)..

I must strongly suggest you review your support policy, even for free users, as it gives us... a very bad impression overall. While I would have understood almost any "no fix" reason due to my "free", community status, this one I did not expect at all.

You can't exactly ask of your users to update to your newest version... when you're a desktop app framework (which brings constraints and inertia in itself, it's not a Saas framework), AND the newest version isn't even officially stable, or its libraries even ready for it.

timunie commented 1 year ago

well, the team is quite small and at some point, the source for v11 had differed too much from current stable. You have to choose one way. 11.0 is really a huge release and many many breaking changes were done for a brighter future. So we hopefully will not get that amount of breaking changes for the next releases. I hope for your understanding.

Again, if you have trouble to update, feel free to ask on telegram or in Discussions.

Last but not least, I think Avalonia has really great community support for free. Not much libs that I know that has such a good support for free. I might be wrong, though. Just my personal impression.