Extensible rendering pipeline

[JeroMiya]

Motivation

Current WPF implementation relies on DirectX/etc... to render vector graphics/UI. There is potentially a desire to use alternative rendering APIs (SkiaSharp, et. al.) for specific purposes, primarily integration within other environments.

Suggestion

As much as possible, refactor rendering pipeline and other platform-specifics to be extensible, allowing for custom implementations decoupled from DirectX/etc..., for example: a SkiaSharp backend instead of a DirectX one.

Potential Questions

Question: "Why not just use Xamarin.Forms? There's already a system there for custom renderers." Answer: Xamarin.Forms implements something similar, allowing for platform specific or custom renderers for specific controls. However WPF and Xamarin.Forms have different scopes and use cases.

[popcatalin81]

Skia is a 2D library (no biggie for most projects if 3D is not supported).

Ideally, there would also be a 3D supporting backend like OpenGL/Vulcan in addition to DirectX for more heavyweight applications which make use of 3D.

[legistek]

I think this is a great idea. I'm just wondering how it might fit in with this statement in the contributing guide:

We also do not intend to accept contributions that provide cross-platform implementations for Windows Forms or WPF.

I'm hoping someone from Microsoft could elaborate on that statement as it relates to something like this proposal.

[Atulin]

We also do not intend to accept contributions that provide cross-platform implementations for Windows Forms or WPF.

As I understand it, anything that would make WPF usable on Linux/Mac/Android/etc. will be rejected. See #48

[Kryptos-FR]

@Atulin I think the proposal here is not to make WPF cross-platform (as in #48) but to make one part of it (the rendering) extensible. It will still have dependencies to Win32 APIs (and others), but at least it could allow some other scenarios as described in the first post.

For example if the rendering was done with Skia, I think you could make some kind of integration with other libraries also using Skia and share the same rendering space (especially important for layouting, Z-sorting, etc.).

[popcatalin81]

While there are no plans for WPF currently to become cross-platform, Still, I applaud this effort from WPF team, making the rendering pipeline extensible will make the cross-platform goal closer to being achievable sometime in the future.

[JeroMiya]

I'm not part of the WPF team, just a user. And yes, as kryptos said, this proposal isn't specifically to make wpf cross platform or increase the scope, but rather just good architecture grooming to help facilitate other use cases and integrations within or near WPF's current scope.

[grokys]

Avalonia currently has this, for example:

https://github.com/AvaloniaUI/Avalonia/tree/master/src/Skia/Avalonia.Skia https://github.com/AvaloniaUI/Avalonia/tree/master/src/Windows/Avalonia.Direct2D1

Just if anyone is interested in any prior art.

[vibeeshan025]

This is not about making WPF cross platform, but this helps WPF moving to New rendering technologies, like from d3d to Win2d. This requirement makes sure that WPF always going forward.

[dotMorten]

... moving to New rendering technologies, like from d3d to Win2d.

Win2D is based on Direct2D which is essentially the same thing / subset thereof. It's nothing new - just a simplified version of it, just like what WPF is

[the-black-wolf]

We also do not intend to accept contributions that provide cross-platform implementations for Windows Forms or WPF.

As I understand it, anything that would make WPF usable on Linux/Mac/Android/etc. will be rejected. See #48

I agree with this, its in line with MS's behavior so far. WinForms and WPF are essentially the last bastions that keep us exclusively tied to Windows platform and there are many desktop based LoBs that cannot be ported to web simply because they would be too slow or are too specialized for local processing (shocker, web is not a solution to everything). For web, they lost server market so they have to play nice with asp.net core in some vague attempt to keep asp.net ecosystem from bleeding into competing archs. Asp.net core also gives them upsell capability for Azure, something that desktop does not. But on desktop they still reign, and they have no real competition (Avalonia porting is a nightmare due to a really weird choice to HTML-ize XAML). Bottom line, and especially with Steam Proton now being a thing ;), if Microsoft released Linux renderable WPF I would permanently remove Windows 10 and switch to Linux as primary-and-only (and I know I am not the only one), we would also switch all LoB workstations in the building to Linux. There would be some work to port .NET Framework to .NET Standard, but we already accepted that as a contingency. They know that, so you will not see a port of WPF or anything being done to ease it until someone expends time and money to make a threatening competing product. The only thing that would happen is that, in light of them slowly suffocating .net framework, we get forced into migrating into .net core 3.0. We will also see a continuation of marketing push to degrade LoBs into UWP or move them to web/Azure. For the same reason, you will also never see them publish the entire WPF codebase here. This project (aside from a slight educational purpose) is just a decoy, a "proof" they play nice with open source, while in reality they do not. Publishing that would allow someone (maybe even us) to fork it and rebrand it and replace rendering so it works on Linux. So thats my 2 cents, and I honestly believe everything else they say is just excuses so they don't come off as totalitarian. Prove me wrong by publishing the entire WPF and I'll apologize.

[legistek]

Don't you think that's a wee unfair (and paranoid)? You can't expect a for-profit company to do something that's against its long term interests. I hardly blame them for not wanting to put THEIR resources toward making Linux or MacOS better competitors to Windows.

(Also if they were afraid of other platforms why buy and prop up Xamarin? Xamarin Forms is more robust than it's ever been.)

I do wish more of the WPF source would be released, and faster. For this project to work the vast majority is gonna have to be released sooner or later. I have no reason to suspect it's being held back though. The source has been public for years; it's not like they're protecting any secret techniques they don't want us to know about.

[the-black-wolf]

Unfair and paranoid? No, I don't think it is. I have no beef with proprietary software, I actually prefer paying for software, then I always have someone to yank if I have problems with it. I also like FOSS because I can pay for support and be supported. What I do not like is these zombie projects, only "free" in name so they wouldn't have to support it (please don't call this half-baked forum-like issue tracker a "support"), incomplete copies of unimportant code base that you cannot fork and rebuild yourself and use without being at the mercy of the maintainer posting builds of hidden files. A crippled unusable collection of source files, which you already had, btw, from debug source servers.
So, no, I don't expect a for-profit company to do something that's against its long term interests. Which is the primary reason for my rant and why you will never see a complete WPF project here while Windows holds desktop market. Nor will you see any effort made by them to decouple WPF from Windows and be usable outside Windows. And I have every reason to firmly believe it is being held back on purpose. Your belief that for this project to work it would need all the files is correct, however, imho, their intention was never for it to "work".

Btw, since you mentioned it, the only reason they propped up Xamarin was to get developer ecosystem to write cross-platform mobile apps which would also be UWP apps (since they didnt get enough traction from devs to make UWP specific builds), in a last-ditch attempt to increase app count for Windows Phone to save it. Key onus being on "mobile". When the platform died, I half expected MS to terminate Xamarin in cold blood like it did Silverlight. I was actually relieved to see MS yielded to Android and allowed app development for it, it saved Xamarin.

[popcatalin81]

The only reason I want cross-platform WPF is to target AMR based SoC devices. I couldn't care less about Windows - Linux wars and who dominates who. AFAIK I'll still be using windows and still be releasing software for Windows, but extending into ARM SoC devices space without investing into a completely new (different from WPF) technology would be really nice to me!

[the-black-wolf]

You do understand you will never be able to do that even if they make cross-platform WPF? If what you say is the only thing you want then Xamarin/Forms is what you want, WPF only has purpose on Desktops.

The main reason they "invented" UWP in the first place as a dumbed down little brother of WPF is because thermally challenged SoC devices do not have enough umph to run full fledged WPF. They tried. On multiple fronts (including an abandoned OS experiment called Singularity), and they failed repeatedly. So they stripped WPF of everything that puts any pressure or power requirement on the system and they got UWP. Which in itself is not such a bad thing until they tried to force it on desktop as well.

I do not care who dominates who either, it never amounts to money in my pocket, we just want to be free from MS control because we lost faith in partnership we had with them but we cannot go full monty on the web to do it. On a personal, non business note, I actually want off Windows because they annoy the hell out of me with their babysitting arrogance and failure to yield on who paid and owns my computer. And I want to do it before they force everyone onto Windows S with a mandatory update, which I am sure is their planned endgame on Windows.

[popcatalin81]

You do understand you will never be able to do that even if they make cross-platform WPF? If what you say is the only thing you want then Xamarin/Forms is what you want, WPF only has purpose on Desktops.

The main reason they "invented" UWP in the first place as a dumbed down little brother of WPF is because thermally challenged SoC devices do not have enough umph to run full fledged WPF

Except I'm already doing it at 30 FPS in Windows running on Atom SoC, with considerately above average graphics complexity. I would like to use the same app (preferably) on ARM SoC (similarity powered) one day.

Properly optimized WPF is very fast, in my experience I will not need to go to UWP for this reason.

[the-black-wolf]

I have not gone too deep into JIT optimizations, haven't done assembler work in 15 years for sure :), but I suspect a lot of problems they had/have on ARM comes from whatever technical or talent problems they have in making a superior JIT for ARM. x64 JIT is an engineering marvel, my hat is off for that, it takes a lot of work and skill to make a GC based bytecode compiler which takes full use of processor abilities and produces native code on par with C/C++. You just have to compare MS and Mono JIT performance to see what I speak of. But for some reason, the ARM JIT never got that TLC, or there is something in ARM architecture which prevents that level of optimization (though it still lacks heavily behind ARM C++ code, so C++ optimizers seem to know/use something they don't). Either way, this core level subpar performance kills every abstraction level above it, including potentially WPF. A cynic in me is telling me they also do this on purpose, if they wanted they could buy talent from an ARM C++ compiler company, or even a company, but then they would have ARM servers and microservers to compete with too on thir own platform. But all this is academic wishful thinking. If they made WPF that is decoupled from Windows and ported to ARM SoC it would also be portable to Linux/Mac. And they will not do that. So your need also falls victim to their long term agenda, just as mine.

[Kryptos-FR]

If you could just remove all the conspiracy theory bits from your post, it could almost be readable. Some of your points could be interesting but you lose credibility every time you talk about some supposedly hidden agenda or pretend to know ow everything about Microsoft business(es) and what decisions/tradeoffs they might have done. The point is, neither you nor me know about all of their decision making. And while it can be entertaining to speculate about it, it's just not the right place. Please keep an objective view.

[the-black-wolf]

Its not a conspiracy theory, conspiracy theory would be something like "Aliens don't like WPF, so Vatican applied pressure to MS". What I said is perfectly in line with a mature tech company finding a new profit venue (service delivery) and redirecting all efforts in that direction, abandoning what they consider less profitable venues (desktop) and attempting to wall garden it to create new revenue stream from it (seeing how no new Windows versions means progressive reduction and cessation of future Windows license revenues). Everything they do in .NET now is trying to herd people into core and from core into Azure. We are not worth anything to them now unless we pay continual fees for cloud services. Where do you see the conspiracy theory here? You name call me because I refuse to align with their revenue stream? While we do not know 100% what their agenda is, a lot can be observed from their behavior, what they prioritize (and in what way), release tempos of various products and services as well as just applying common sense to what comes out of marketing and what they try to sell you. You don't have to be a genius, you just have to not be a fanboi and blindly trust every word that comes out of them. And also observe the best interest of yourself and your firm and not be a sleeper agent for their sales team. On this topic, this project was posted some time ago, updates are minuscule, project is unbuildable and unusable, they say they will post more, I call BS on this expectations that it will ever be complete, and all that seeing how nothing is really preventing them from posting it right now this second. The thing is, its exceptionally easy to prove me wrong, they just need to post the source, nothing more, nothing else. I build WPF and include it my project and it works, I apologize. You can apply whatever label you want to me personally, nutjob, conspiracy theorist, linux fanboi, ms hater, whatevs. But you still wont have buildable WPF, let alone forkable one. So, I also call BS on MS marketing flavor of OSS. Whatever this is, its not OSS. Not in practice and most certainly not in spirit.

[popcatalin81]

@hades-incarnate

Please tone down those walls of text.

Try to separate your phrases into distinct paragraphs To make them easier to read.

Thanks

[JeroMiya]

Closing this issue for now as unlikely to be implemented, and the discussion is becoming counter-productive. I recommend people use Avalon or maybe even Xamarin.Forms in the meantime for the use cases that might need an extensible rendering pipeline in a XAML framework.

[legistek]

You will never see a complete WPF project here while Windows holds desktop market

@the-black-wolf almost all the WPF code is now in the project and open sourced. Looks like you owe Microsoft an apology. ;)

[the-black-wolf]

You will never see a complete WPF project here while Windows holds desktop market

@the-black-wolf almost all the WPF code is now in the project and open sourced. Looks like you owe Microsoft an apology. ;)

Hmm, what did I miss? Has something happened that can switch this issue and #48 to resolved? I don't see it in the commit log. So lets not play this game. Please provide commit hash, or it didn't happen.

Also please note that "almost all" is not "all". Second placed person has "almost won" but has not "won", only the one who "won" takes the gold. See the difference? This applies to life in general, especially in situations where "almost" is the key part that cannot be ildasm-ed and makes all the difference.

[legistek]

Please provide commit hash, or it didn't happen.

https://github.com/dotnet/wpf/commit/ae1790531c3b993b56eba8b1f0dd395a3ed7de75

the key part that cannot be ildasm-ed and makes all the difference.

You obviously don't know what you're talking about. For anyone who's actually interested in having a productive conversation rather than just hating on Microsoft, the fact is that everything the community needs to make a cross-platform WPF implementation is now open sourced and freely available. I for one am very excited.

[the-black-wolf]

You obviously don't know what you're talking about. For anyone who's actually interested in having a productive conversation rather than just hating on Microsoft, the fact is that everything the community needs to make a cross-platform WPF implementation is now open sourced and freely available. I for one am very excited.

So, you will seriously make me do this. Ok, no problem, don't complain later. Lets chat about the actually important things, which make WPF unique in regards to, say, Avalonia, stuff that makes 3rd party components (potentially, if they are not written dirty with GDI hardpoints) usable and those things would be MIL for start. Try to follow up; make any control, or piece of code that creates primitives, using say DrawingContext, try burrowing inside, and you will discover that it in turn internally results in using a specialized rendering context which will, in turn, simply dump all the primitives' metadata into MIL, which, we suspect from PR, does its magic on DirectX surfaces making a generally cumbersome UI engine actually very fast and usable. No MIL, no joy, no way to recode it to OpenGL/Vulkan, no threat to Win desktop. I see no MIL code in the commit. Everything else can be found in source servers and can even be ildasm-ed 😇. Heck, Avalonia has it all and even has an extensible rendering pipeline. The whole reason we are even discussing all this is Avalonia's choice to make a breaking change in resources, making direct ports of all legacy WPF apps and use of cleanly written 3rd party components impossible. If they followed the WPF specs Avalonia would win, even with some performance losses. Why they chose to do this is theirs to know, and they owe no one any explanation, just like MS does not owe it for not releasing MIL. But, then lets not call things what they are not.

As for name calling, I say better a hater than a fanboi. That ship of trust has sailed, it will take a lot to bring it back.

[legistek]

So your complaint that they haven't yet open sourced the native DLLs constituting the composition engine that uses DirectX (such as wpfgfx)? What good would that do on other platforms anyway?

You'll note that wpfgfx is in the roadmap to be released. They have slowly but surely been making their way through it. We have no reason to believe they won't release it.

Regardless, you have the specs for the native composition engine from, I believe, Composition.cs in PresentationCore, if not other places. So if you wanted to replace wpfgfx with your own composition engine based on OpenGL or whatever you want, you can. I for one think replacing it with something that uses DirectComposition would give great performance improvements.

Likely the reason Avalonia departed from WPF syntax was copyright reasons. Yes, it's true you could have gone to the .NET Framework reference source online for years and seen exactly how it all worked (you don't even need ildasm). But it was not licensed for copying, modification or distribution. Now it is. So a new "Avalonia" could come along and copy the WPF syntax EXACTLY, not to mention huge chunks of the code - almost everything in PresentationFramework - but use their own rendering engine. I hope, and in fact am sure, that will be done in time.

[grokys]

To clarify here: Avalonia not following WPF's API had nothing to do with copyright.

What happened was: Avalonia was originally intended to reverse-engineer WPF, but I got to CollectionView and disagreed strongly with its API and semantics. I just wasn't having fun reverse engineering WPF and so I decided to start messing around doing things "my way" in a project called Perspex.

Perspex took off and was eventually renamed to Avalonia due to a trademark on the name Perspex. The original Avalonia languished and was eventually archived. You can still find it here though: https://github.com/grokys/Avalonia.

If I could go back in time, I would definitely follow WPF's API more closely (though not everything, because some of it is batshit crazy and other parts could never be made x-plat) but: hindsight and all that. I was the only person working on it, and had no idea anyone else would be interested. What I can say is, the alternative to "doing my own thing" with Avalonia was just giving up and working on something else.

The good thing is that we did follow WPF's API for DrawingContext pretty closely, which means our rendering backend might be usable in some form for WPF if someone wanted to do the work.

I know that this is only tangentially related to this conversation but I wanted to set the record straight.

[legistek]

Oh hey thanks for clarifying that! Although now I'm mad as hell you didn't follow WPF. ;) Just kidding. It was the right decision, because based on the Oracle vs. Google decision you could have had an issue copying the API syntax.

I don't suppose you're interested in rebooting Avalonia using the WPF APIs now? Second time's a charm!

[grokys]

We've definitely talked about it. I suspect it wouldn't be easy though. There's just so much of WPF's API that's hardcoded to win32.

Edit: not "rebooting Avalonia" as such. You'd be asking us to just throw away the last 5 years of work. We've talked about having essentially a "compatibility layer" where some of WPF's API runs on top of Avalonia.

[legistek]

Nah I wouldn't want you to throw anything away. :) Even if we could have identical XAML syntax that would be a game changer.

[JeroMiya]

To be perfectly honest, if the goal is to create a truly cross platform UI framework that renders it's own controls (i.e. not rendered with platform native controls like xamarin.forms or react native), the most efficient use of .net open source developers' time would be to work on a new framework or preferably port an existing framework designed from the beginning for this purpose. My vote would be a .net flutter port.

This would eliminate all windows dependencies, weird API decisions from early WPF we'd be stuck with, and a more modern UI and state model like react and flutter.

[jmacato]

@JeroMiya You've basically just described what Avalonia is. It renders its own controls like WPF does and is not tied with the cruft and tech debt that WPF currently has.

If it needs to be ported to a platform (that dotnet supports anyway) then you can implement the platform specific rendering and windowing layer to a standard set of interfaces that Avalonia provides.

You can do whatever kind of UI state paradigm you believe in (MVVM, Elm, etc.) with it since it doesn't restrict you to a pattern.

Anyway, In my (biased of course) opinion, if one is looking for a cross-platform version of WPF that behaves and feels like it then Avalonia is your best bet.

[JeroMiya]

@jmacato Yeah, although I'm not familiar with Avalonia specifically, it sounds like it has a lot of the core ingredients and would be a good candidate, but I was thinking something less on the WPF/Silverlight/XAML data binding model and more like the React/Flutter/Blazor model. It seems like it's harder to retrofit a react or flutter-like shadow DOM model over a mutable two-way databinding framework than the other way around.

[legistek]

You can do data binding in Blazor. It's not part of the official framework but it can be added easily. Data binding just requires reflection and property change events.

It's probably possible to do with Javascript too (ie React) but I've never tried.