Let's consider adding a faster, modern rendering engine for custom 2D and Text rendering like Skia or Direct2D/DirectWrite

[KlausLoeffelmann]

Since this topic is one of my personal favorite topics, which I deal with as a hobby in my spare time, I have been experimenting with GDI+, DirectWrite/Direct2D and SkiaSharp a bit lately. While we've made significant improvements around GDI/GDI+ memory consumptions and efficiency, we hear more and more feedback that the WinForms community finds GDI+ too slow in its (rendering) core for modernizing older Framework Apps and moving them to .NET. And it makes sense: Customer's use more and more 4K monitors (often more than 1), and GDIPlus takes much longer time to render bigger area.

So, here are a couple of thoughts as a guideline for questions and discussions, which I think would help to form a better picture around this topic.

• Do we need to consider this at all or does GDI/GDI+ suffice after all for the custom drawing things we want to do in WinForms?
• DirectX/Direct2D is native to Window. Is that good or bad, is this a pro or a con argument?
• Same question for Skia(Sharp) which is NOT native to Windows. But since it's the render engine for Google's chrome, it's been used by Edge browser indirectly.
• Skia has an extremely easy way to render into PDF, which is an extreme plus for the typical WinForms (LOB Apps) audience. I am not aware that DirectX/Direct2D allows this equally easy.
• DirectX/Direct2D is the base for rendering WinUI. Should we see XAML-Islands for WinUi 3 in the future (and I am not at all implying that we won't, just to be clear!), there might be easier migration options for content rendering going forward.
• Skia would introduce external dependencies, which we might not want or like. Or do we? For this, the next issue might be relevant:
• Is this (no matter what technology we would go for) something which we want to have as an add-on or deeper integrated into WinForms? Would we like to - for example - have a control like the DataGridView, which is completely rendered by GDI/GDI+ and not a wrapper around some Win32 control, refactored so it would be rendered by that modern rendering engine?
• Are there any additional rendering engine options we should include in the discussion?

Here is a good document to get the essentials of Skia in the context of GDI/GDI+: https://www.chromium.org/developers/design-documents/graphics-and-skia

Here are the (visual) results of a .NET 6 app I wrote and experimented with.

DISCLAIMER: Please note, that I did not use the existing SkiaSharp WinForms controls, since they pull in .NET Framework dependencies and the OpenGL version of the Skia WinForms control is based on a rather old (Framework) version of the OpenTK's GLControl. While reimplementing the SkiaSharp controls based on the latest version of OpenTK (5.n) works for most of the things I tried reasonably well, I had TONS of problems when rendering circles, ellipses (which are internally seem to be converted into arc-based paths, which I think is the problem) and the alike and could only make those work on one of my several test machines, and also only with a BIG memory leak.

This demo just animates a few hundred Circles in different (transparent) colors and bounces them off the boundaries:

This is GDI+:

This is Skia: https://user-images.githubusercontent.com/9663150/147891114-bf532588-a87b-4436-a7e3-fc861272191c.mp4

This is Skia based on OpenGL: https://user-images.githubusercontent.com/9663150/147890994-74c26c8a-4a33-496c-a125-85f8c2c9f2fc.mp4

[kirsan31]

@KlausLoeffelmann do you forget to add second (Skia) gif/video?

[KlausLoeffelmann]

No. They are currently all just too big as GIFs, so I couldn't add them. I have to recapture them with Camtasia and edit them.

[paul1956]

At least a couple of the WinForms Apps I work on regularly hanged for extended periods of time rendering large datasets and had to be recoded to smooth out the behavior it, would be nice if DataGridView just worked, which technology used to me is an implementation decision.

[KlausLoeffelmann]

@paul1956: Can you elaborate on that? Is this a rendering problem in the DataGridView, where it takes too long to actually bring the representation for the already present data on the screen, or is the problem to get the data in time to then render without interruptions?

[paul1956]

I can only describe the symptom. When I have a lot of rows in DGV, when filling the grid it tries to update the display after each row is added and that is very slow. When it finishes, which could take minutes, scrolling even 1 raw takes almost as long. I now only load what is in view but there is a lot of manual management to get that to work. I think the answer is both, if the grid is full scrolling is an issue, is the grid is visible loading the grid is extremely slow, one workaround is to make the grid invisible while loading.

[2mik]

Currently WinForms look like an abandoned child. They used to be almost bugless and high quality, but not now. Hope this situation will change.

[RussKie]

No. They are currently all just too big as GIFs, so I couldn't add them. I have to recapture them with Camtasia and edit them.

Screen2gif can save as mp4, which can be uploaded here.

I can only describe the symptom. When I have a lot of rows in DGV, when filling the grid it tries to update the display after each row is added and that is very slow. When it finishes, which could take minutes, scrolling even 1 raw takes almost as long. I now only load what is in view but there is a lot of manual management to get that to work. I think the answer is both, if the grid is full scrolling is an issue, is the grid is visible loading the grid is extremely slow, one workaround is to make the grid invisible while loading.

Please open a new issue with a repro.

[KlausLoeffelmann]

Please open a new issue with a repro.

Yes, it would be also really interesting to know what you do to mitigate!

[willibrandon]

I am very interested in easier ways to render to PDF.

[nathan130200]

If skia is used to render winforms, maybe can be turned into multiplatform, since all rendering stuff will be on multiplatform library. I know that exists Maui, but maui have lot of boilerplate, and winforms is fast, simple and easy to develop.

By myself i prefer winforms and raw wpf instead of maui.

[antonfirsov]

I wonder how open would the WinForms user community to a solution that is built around Maui.Graphics. In theory this lib should pull in only a small friction of the Maui boilerplate.

Pros for this approach:

• Avoid introducing yet another Drawing API to the .NET ecosystem
• There may be some synergies between the Maui.Graphics efforts and the goals described by the OP, this could help reduce duplicate investments.

/cc @mattleibow

[KlausLoeffelmann]

I will certainly take a closer look!

[KlausLoeffelmann]

I wonder how open would the WinForms user community to a solution that is built around Maui.Graphics. In theory this lib should pull in only a small friction of the Maui boilerplate.

I took a look at that this weekend. In fact, I liked it so much, I think we should really go for it. I for once will certainly push for it, So. While there is a GDI implementation for Maui.Graphics for WinForms, I think, we shouldn't do GDI. We should do Direct2D/DirectWrite. I started implementing a prototype, and could make a few basic shapes work.

It is by FAR the most stable and fastest way to do.

And since there are also Skia implementation of Maui.Graphics, (and without ever actually trying this), I would say you can take an IDrawable for Maui.Graphics and also could produce a PDF with it.

[antonfirsov]

While there is a GDI implementation for Maui.Graphics for WinForms, I think, we shouldn't do GDI. We should do Direct2D/DirectWrite.

In an ideal world that improved Direct2D should go right to the Maui.Graphics repo. Being you, I would open an issue against them so this topic gets more attention :)

[KlausLoeffelmann]

Give it some time to cook. I'd also like to discuss this with the team. But yes, on first glance that sounds most reasonable. Here by the way is what it looks like: (I've been too lazy to make a Camtasia this time.)

https://twitter.com/loeffelmann/status/1483242987991699458?s=20 (Also @JeremyKuhne, @merriemcgaw, @RussKie)

[AraHaan]

I wonder how open would the WinForms user community to a solution that is built around Maui.Graphics. In theory this lib should pull in only a small friction of the Maui boilerplate.

I took a look at that this weekend. In fact, I liked it so much, I think we should really go for it. I for once will certainly push for it, So. While there is a GDI implementation for Maui.Graphics for WinForms, I think, we shouldn't do GDI. We should do Direct2D/DirectWrite. I started implementing a prototype, and could make a few basic shapes work.

It is by FAR the most stable and fastest way to do.

And since there are also Skia implementation of Maui.Graphics, (and without ever actually trying this), I would say you can take an IDrawable for Maui.Graphics and also could produce a PDF with it.

Will it be made with themability of everything in the ideas (like for example the default scrollbars on a panel for example being themeable, or other controls when controls inside of it span past the defined size of it)?

Because of the lack of this in Windows Forms, I been considering myself to make a "somewhat fork" of Windows Forms that uses TerraFX to call directly into Windows APIs that then in turn would need to be debugged to allow such thing for all controls when it sees that a scrollbar should be shown so then it manually draws (somehow) the way they want Windows to draw them.

It might sound like it's a bit much, but modern applications need to be able to theme themselves entirely and look nice at the same time. Paint.NET is a prime example of where manually drawn themes for everything just makes the program look nice, however a lot of Paint.NET today has code to do such thing which really should be part of a Desktop Framework that ALL Desktop Applications should be able (they just define a subclass of a generic Theme class and then they pass it into an application level configurator that then would be used to assign that theme to all windows and can be used while the Window is shown to switch the theme as well (have it invalidate the entire window for repainting)) to use if they actually want to without using WPF or Maui (either because they do not care for WPF or they just cant port it to WPF for a reason, or they do not care for crossplatforming their Desktop Application that is Windows only on their code as well making Maui a waste of an effort anyway).

While some people might use code in their Windows Forms application which is 99.9% cross platform, not everyone is that lucky and might have only 0.1% that is cross platform.

[sterenas]

Sorry about my confusion here but when you say “Winforms” does that include VB.NET also? That’s be awesome! Thanks!

[KlausLoeffelmann]

Will it be made with themability ...

No. That's not the scope of this feature idea. Also, at this point it is a discussion, not more.

The whole topic is "just" around an alternative to the rather slow GDI+ rendering.

That said, theming is something which is rather on top of our modernization list. Not in this context, though.

And yes, this would be supported in VB. I don't see any reason why not.

[2mik]

If it's decided to get rid of GDI+, may be it's possible to make WinForms cross-platform as Mono did?

[KlausLoeffelmann]

If it's decided to get rid of GDI+, may be it's possible to make WinForms cross-platform as Mono did?

We're not getting rid of GDI+. It'll be always in WinForms. And no, the decision for this has nothing to with that. WinForms is a wrapper around W32 controls. They are rendered by Windows.

[kirsan31]

I'm worried... Maui.Graphics is an experiment now.

There is no official support. Use at your own Risk.

I can'nt find any plans/roadmap on it. 🤔

[dax-leo]

WinForms is still used a lot across many large companies. I work for one such company (Ericsson) and we use WinForms for all small-medium size tools (next to C++/QT). It's the most productive framework that came out of .NET. I've been using .NET since 2002 and God knows how many times I had to create quick application in WinForms (nothing against WPF), but WinForms always felt more friendly to me.

But it desperately need update in form of Hardware Rendering (OpenGL or DirectX). If anyone disagree I challenge that person to use WinForms on 4K monitor. Load datagrid with 10-15 or more columns and try to scroll down. I basically breaks down due to very slow SW rendering. This mean in future when most monitors out there will be 4K, without hardware rendering support WinForms might become obsolete and unusable. I think this should be No1 feature on .NET Team priority list.

Hardware rendering in Winforms is nothing new. DevExpress upgraded many of its WinForms components to DirectX, while company called Nevron is using OpenGL (there are some opensource attempts but they all kind of suck). Devexpress WinForms with DirectX support have unbelievable performance. Also, don't forget that with DirectX there is also significant memory utilization benefit.

Normally I don't use third party components like DevExpress due to large dll (+pre-JIT) overhead (except excellent Krypton toolkit - which is opensource). So having native support for DirectX or OpenGL in WinForms would make it crazy good.

Please consider this option! ... and please don't abandon WinForms !!!

[AraHaan]

I been considering making an DirectX experimental GUI framework, and if it's a success may consider porting it to the winforms codebase.

Also I agree, recently I been making all of my libraries crossgen2'd (R2R) with success to avoid slowdows like the way you described. However it comes at a cost of disk space I think though.

[lucapivato]

https://www.jitbit.com/alexblog/300-systemdrawing-vs-skiasharp-benchmark/

[antonfirsov]

https://www.jitbit.com/alexblog/300-systemdrawing-vs-skiasharp-benchmark/

The code generates a 120x80 thumbnail from a 500kb picture

What does a 500kb picture mean? What format? What resolution?

(1) In any case 500kb is small, not representative for typical workflows today, dealing with images coming from 12MP, or even better phone cameras. (2) the benchmark doesn't stress concurrent execution, therefore it's useless to make any conclusions about high-load, scalable web scenarios.

Sure, System.Drawing might be still okayish for some desktop apps, but the blog post is talking about a web app. In that environment, it was a terrible choice in 2017 already: https://photosauce.net/blog/post/5-reasons-you-should-stop-using-systemdrawing-from-aspnet

[lucapivato]

I didn’t write it. Just found it interesting.

Trying to simply draw underlined or wrapped text with skia seems to be a complicated task. Google’s blog post about underlined text is that it’s a controversial feature… always trying to save developers (many of which have naturally more “field” experience) from themselves is quite annoying and patronizing, again in my view.

Seems to be a nice library to draw bouncing balls or large landscapes, but it fails miserably measuring text and drawing text. That is of course in my experience. I didn’t find any code anywhere, other than generic claims of “modernity”, that works correctly.

We have large apps in production using gdi+ running on azure or local servers transacting millions of requests per da without any issue from gdi+. I could never reproduce the claims of doom in any reasonable scenario. Again, just my humble experience with real-life large apps in production.

[lucapivato]

a terrible choice in 2017 already:
https://photosauce.net/blog/post/5-reasons-you-should-stop-using-systemdrawing-from-aspnet

Just thought to give a try to the 5 reasons with some simple tests:

• Created 1M fonts each in 4 concurrent threads (on a 12 core machine, each thread has a random Sleep call to block) randomizing the family name and the size to prevent some internal caching. Uses about 0 GDI handles overall, unless you call ToLogFont(), which is expected. The GDI handles do increase by about 50 but drop quickly. Which is what happens on almost every process on Windows.
• Drawn about 100,000 bitmaps in 100,000 Graphics objects created from new Bitmap objects and drawn a simple underlined text in 4 concurrent threads, each had a random Thread.Sleep to block the thread. There was not blocking or process-wide lock.

Overall I see a high level of concurrency, low memory usage, little or no GDI handle usage and a very easy to use, well designed API.

In general, blanket statements about something being terrible in software are usually wrong for one or another requirement. Some things may be terrible for some requirements and good for others. It's usually better to build specific test cases and check it out to see if it meets certain requirements.

And that's what I'm doing with SkiaSharp, wrapping the 20-year old skia library. For my requirements, not for every other developer in the world. I can easily rebuild a System.Drawing API based on SkiaSharp except for some few key problems:

• Text rendering is off
• Text wrapping non-existent
• Simple text decoration has been removed and requires a ton of code adding points of failure
• Text measuring is wrong, it's even different from Chrome itself (I will have to dig in Chromium to see what they do, not a good sign for the usability of an api surface)

For simple images it works well (although a bit slower than GDI+ in Windows and uses a good amount of memory) and I can rebuild a System.Drawing API based on it except for the text stuff. In that case I'll try to consolidate the random code I see around that draws lines under the text and tries to wrap blocks in a meaningful way.

[TheDecryptor]

Another point in favour of Direct2D/DirectWrite, it has special support for RDP and can send just the drawing commends over the network to let the client system render them instead of sending a rasterised bitmap (Which would be the case with Skia)

[nathan130200]

I still prefer winforms instead WPF for fast building desktop application. I can quickly drag'n'drop controls and 🥳 TADA! i have an desktop application running. I prefer WPF when requires lot of complexity or when just GDI+ cannot handle.

For example i was working an pseudo 2d cell game simulation just for testing skills on 2d games, but its impossible to use winfors.

Each rendering when cells count increases makes GDI+ rendering very very slow untill an single frame took too long to render.

Ok .NET have now MAUI, but sometimes fast and simples solutions are better than just "fancy apps". But the problem is here people think that better apps are just because they are beautiful. This is WRONG, applications must work, be optimized and winforms can provide this for us.

Also some components like DevExpress did with DirectWire/Direct2D rendering their components/controls are amazing and well optimized. This is my favorite set of controls.

[antonfirsov]

@lucapivato I agree that text rendering is one of SkiaSharp's weaknesses, and my intention wasn't to argue for SkiaSharp, but mostly to point out the flawed methodology in the mentioned blog post.

In general, blanket statements about something being terrible in software are usually wrong for one or another requirement.

https://github.com/dotnet/winforms/issues/6459#issuecomment-1231979418 without further context suggested that System.Drawing is a winner choice for server-side thumbnail making, which is simply not true (see below), thus the strong reaction in my response. Otherwise, I would like to correct my blanket statement. Various libraries/stacks provide very different performance and "feature richness" for various use-cases. It can be a totally valid conclusion that System.Drawing is a good choice for Windows-only "kinda-more-advanced" text rendering today, even on server, assuming the app code is well-written and avoids handle hell. I hope that the massive work in SixLabors.Fonts will soon bring a cross-platform alternative for server/off-screen use.

OFF - Since this issue is primarily concerned with WinForms 2D Drawing and Text Rendering

- FYI [some benchmark work that compares various libraries for end-to-end image-resizing at scale](https://github.com/saucecontrol/PhotoSauce#magicscaler-performance). Note that ImageSharp numbers under "MagicScaler Efficiency" are outdated, the library consumes much less memory since 2.0. - I also did my own benchmarking based on a modified/extended [variant](https://github.com/SixLabors/ImageSharp/tree/main/tests/ImageSharp.Benchmarks/LoadResizeSave) of MagicScaler's benchmarks, and my results confirm that System.Drawing is not doing a great job at this particular task: ``` Downscaling ~13MP images on a 10 Core i9-10900X + 64 GB RAM. The entries represent execution time milliseconds, lower is better. Scalability is a quotient: execution time with Parallelism=20 / execution time with Parallelism=1: ``` | | Parallelism | | | | | |-------------------------------|-------------|------|------|------|--------------------------| | Library | 1 | 5 | 10 | 20 | Scalability = P(1)/P(20) | | | | | | | | | SystemDrawing | 8745 | 3129 | 2700 | 2707 | 3.2 | | ImageMagick | 10239 | 2751 | 1921 | 1494 | 6.9 | | Skia | 6398 | 1778 | 1086 | 838 | 7.6 | | ImageSharp | 4835 | 1401 | 748 | 611 | 7.9 | | NetVips | 1883 | 532 | 346 | 322 | 5.8 | | MagicScaler | 1936 | 600 | 447 | 358 | 5.4 | | Skia - DecodeToTargetSize | 2241 | 634 | 390 | 301 | 7.4 | I understand that when talking about advanced drawing and font rendering, this use-case might seem dumb and unimportant, but it's bread and butter for many web applications and CMS-es, so I think it's not great when blogs spread superficial information in this field.

[AraHaan]

Another thing I do not like about System.Drawing: When needing to zoom for example a 1,200 x 1,200 px image by 2000~3000% you will find that System.Drawing cannot handle it (I use the resize method on images and calculate the new size using original size and the % amount). It has too many limits for uses like that.

[elachlan]

I use the third party DevExpress libraries for our LOB apps. They have introduced DirectX rendering for many of their controls, the performance of their GridControl is impressive.

I'd love it if this was native to Winforms. Devexpress currently have a DirectX based Form where you can place DirectX based controls, but its designer experience is lacking.

[merriemcgaw]

That's great input for us. This would be a longer-term investment that is definitely worth considering. I don't know that it will make .NET 8 given our current priorities but I do NOT want to close the idea down. It's something I'd like to keep revisiting until the priorities are such that we can tackle this for you 😄

[nathan130200]

This could bring an possible rendering layer API to people use/implement many renderer and making winforms multi-platform 🎉

[sgf]

I think this is difficult to achieve because winform is essentially win32 api. And I'm not sure whether d2d is the rendering basis for windows native win32 windows. In addition, it should be noted that the design and performance of d2d itself are not excellent. They are just slightly better.