Port System.Xaml to .NET Core

[terrajobst]

Filed as a result of the discussion in dotnet/runtime#14862.

Note: this doesn't represent a commitment from us to open source System.Xaml or to even port it to .NET Core -- it's simply capturing the community request to do so.

---

Scenarios

1. WF (Workflow Foundation) - dotnet/runtime#14862
   • However, note that CoreWF is also trying to abstract serialization from Xaml (as it was originally intended). That said, all existing Desktop serialziations are in Xaml and we need it for transition path at minimum (having converters would be another option).
2. Prereq for WPF
   • Note: WPF / GUI frameworks are a big step for .NET Core (which targets server and console apps scenarios today) - it needs full end-to-end plan & commitment first.
3. Prereq for UI frameworks
   • https://github.com/AvaloniaUI/Avalonia - @grokys (see https://github.com/dotnet/corefx/issues/5766#issuecomment-275260663)
   • Eto.Forms - @cwensley (see https://github.com/dotnet/corefx/issues/5766#issuecomment-275282292)
4. ... more scenarios will be added based on the discussion
   • @Mike-EEE's devops scenario - waiting on more details
   • @Kation's non-UI framework scenarios - waiting on more details

[Mike-E-angelo]

HORRAY! Official recognition/acknowledgement! And yes, I realize that those words do NOT equate to "commitment" but this is definitely better than anything that I have seen so far. @terrajobst you are my hero. :heart: :heart: :heart:

[Mike-E-angelo]

@cwensley and @SuperJMN, it would be great to get some dialogue between everyone here. Ideally (from my perspective) everyone should ultimately use one port/version of the new Xaml library and not have different versions/dialects of Xaml out there. Is this even remotely possible? I'm OK with it if not, but I would like to see if we can if so. :)

Also, I am currently on a "working sabbatical (from bossy clients LOL)" and I have 3-4 months slated to cross-platform Xaml alone. I could literally work full time and help out on this during that time if necessary. Anything for the cause (and also, I'm an opportunist for good timing!)

[shahid-pk]

if System.Xaml got open source i am sure the community can port it to .net core according to this project's requirement and design goals, the project maintainers will only need to guide the community. Lets leverage the power of .net community.

[cwensley]

@Michael-DST, I agree having one dialect of xaml is important. However, the implementation of that might be entirely different. Standardizing on a single implementation depends on the community really.

For Portable.Xaml which is a PCL 259/136 port of Mono's excellent System.Xaml (MIT licensed) implementation, with many bugfixes. The idea is to support everything that MS' System.Xaml does (including reading/writing) using the same API, but not necessarily be limited to it.

As an example of extended functionality, Portable.Xaml supports using type converters on items in a list, where System.Xaml uses IList.Add(object) which you'd have to implement using a custom collection to support various types.

[Mike-E-angelo]

@cwensley I claim my ignorance on this... but what is the difference between .NET Core and PCL? I thought they were the same? That is, it was my understanding that if you designed under a certain PCL profile, that it would carry over to .NET Core... this was a while ago (and honestly I was waiting for .NET Core to RTM/W), so now is a good time to refresh. :P

It is my understanding/impression that Portable.Xaml is indeed a port of mono's System.Xaml, while OmniXaml is a rewrite from the ground up that features some new hotness in the parsing arena. Ideally (from my own perspective) it would be great to merge the two somehow, but I am not even sure that is feasible, or even desirable between the two bases. In my mind (without doing the spelunking/analysis) it is basically merging the maturity of the one (Portable.Xaml) with the new hotness of the other (OmniXaml).

[cwensley]

@Michael-DST PCL 259 is indeed compatible with .NET Core, according to this post. I have already included \lib\dotnet as a target in the nuget package for Portable.Xaml, so it should be usable as-is for .NET Core, though I have not tested it.

I haven't had the time to look into OmniXaml, but if it is indeed a rewrite from the ground up it may not make sense to merge the two projects as I'm sure things would have been done completely different. I'm also not sure of the benefit of such a thing if Portable.Xaml is mostly complete as is. I could be wrong though (;

[Mike-E-angelo]

:) Yes, I am exploring options here, and also dealing with my ignorance with really being in an open source world. I do hate the idea of having multiple codebases out there doing the same thing. And especially with the Xaml community being smaller, it would suck having it get fragmented right out of the get-go (isn't that supposed to happen after something gets too successful for its own good?! haha). Anyways, it would be great to hear @SuperJMN's take on this. I know he seemed pretty firm on his new modifications. For instance, the use of MarkupExtensionContext in place of IServiceProvider for markup extensions, which would be a breaking change for anyone wanting to port existing code over to this new world.

[mellinoe]

To clarify, all PCL profiles are compatible with .NET Core. The contract sets and factorings we have now are an evolution of PCL profiles. This doesn't apply to "old-style" portable libraries (compiled against mscorlib, etc.) which aren't immediately compatible with .NET Core.

[terrajobst]

@Michael-DST

@terrajobst you are my hero. :heart: :heart: :heart:

While I highly appreciate your enthusiasm I want to make it clear that anybody can file requests like this. You don't need any sponsorship or acknowledgement from us in order to file issues, asking for work :-) I merely filed it myself to save some time and not be super lame ("feel free to file a bug") :-)

[Mike-E-angelo]

LOL @terrajobst it is OK, my enthusiasm is half tongue-in-cheek and half-serious... MSFT has been basically silent on this issue for nearly a year now, despite numerous votes and extensive conversations.

So, consider what you did the equivalent of sliding a meal under the door of someone locked in solitary confinement for nearly a year. :stuck_out_tongue:

[Edit: after reading that analogy (and seeing it quoted below), I have to say it's terrible and probably influenced by an article that I was reading at the time -- damn you news, why must you influence me so?! A better/relevant one would probably be getting a much needed injection of cash from an angel round after a year of running on fumes.]

[zhenlan]

cc: @Harikm86, the owner of System.Xaml

[rschiefer]

I love XAML. Its a great declarative language for everything from Silverlight (yes, I said it) to Windows Workflow and now to the new Universal App platform. Porting to Core will make it possible to port other technologies that depend on XAML. We need this!

[Mike-E-angelo]

Great to hear your support @rschiefer! Thank you for your input. Please make sure you have other Xaml developers that you know of join this thread and help out with the conversation/direction.

I am also a fan of Silverlight and have been chasing after its next incarnation since 2011 (shameless plug: if you would like to let Visual Studio team know you'd like to see the next, smarter form of Silverlight, please vote and let them know here).

I would like to take a second and make sure that when we say "Xaml" we should aspire for the Xaml system/feature set found in .NET 4.x+ (or "System.Xaml"). Even Silverlight 5's Xaml will do, really.

(Of course, adding in the new improvements found in OmniXaml is desired, as well.)

As long as it is not the "new", bastardized version in UWP that must have been ported over by a team of interns, as we all have been suffering from it since (see the votes that I mention in my post above).

I really even hesitate to call that a "Xaml" system, as it is really XML with some extra token parsing for good luck (which it clearly needs). Fun fact: UWP's Xaml system ultimately uses the System.Xaml assembly during its build process. Mmm... #irony.

Ideally once this port is in place, the UWP Xaml system will be deprecated altogether and use this one instead. We can then put that chapter of embarrassing Xaml history behind us, and much developer rejoicing will be had.

Getting Silverlight back in a newer, better form wouldn't be bad, either. :) :) :)

[terrajobst]

@Michael-DST

So, consider what you did the equivalent of sliding a meal under the door of someone locked in solitary confinement for nearly a year. :stuck_out_tongue:

Fair enough :smile:

[rschiefer]

@Michael-DST

I have actually avoided UWP up to this point knowing a lot had changed and I had forgotten UWP used its own "version" of Xaml. Thanks for reminding me.

I completely agree, System.Xaml should be ported and then we can fix UWP to use real Xaml.

[Mike-E-angelo]

I completely agree, System.Xaml should be ported and then we can fix UWP to use real Xaml.

Sorry @terrajobst, I have a new hero now in @rschiefer. :smile: :smile: :smile:

Thank you all for the discussion. This is super cool to me (but I could just be easily impressed!).

[helmsb]

@Michael-DST @rschiefer

[Mike-E-angelo]

@helmsb YES! LOL haha!

[ghost]

FWIW, the competitive Google's popular Material Design tech for web (Angular.js), apps (Android) etc. is also available open source. Hope this argument help convincing the bosses and boss-like entities.

[galvesribeiro]

@dsaf what you mean?

[Mike-E-angelo]

FWIW I have referenced this issue in an article posted today titled Is the Tide Turning on project.json?, a look at how Xaml is a preferred mechanism for describing .NET objects from both a technical and MSFT business/IP perspective.

[Mike-E-angelo]

Well everyone, how about that Xamarin deal, huh! something tells me that this issue is going to get a lot busier in the coming months. :smile:

[RichiCoder1]

Not necessarily. There might be a good chance what we see UWP's Xaml or Xamarin's flavor of Xaml become some cross platform library. Which might mean neither a port of System.Xaml or even a purely managed Xaml implementation may happen.

[Mike-E-angelo]

Xamarin's Xaml system is closed/internal though and nowhere near as mature as System.Xaml. The code itself is rather flimsy (an acknowledged concern on their end -- actually using that is one of the reasons it is closed/internal). Additionally, it is tightly-coupled to BindingObject which is prohibitive if you are wanting to do POCO serialization, etc.

And as outlined above, UWP's Xaml system is t-e-r-r-i-b-l-e. Unless there is something that you know that I do not know about. :stuck_out_tongue:

Can you explain what you mean by "purely managed Xaml implementation"? FWIW, my expectation here isn't exactly a direct 1:1 port of System.Xaml but a "new" System.Xaml (Microsoft.Xaml?) library that takes all the goodness from System.Xaml, Xamarin.Core.Xaml, and ... well, I would say UWP's Xaml, but... :wink:

[RichiCoder1]

I've heard that internally there's a preference for UWP's Xaml over WPF and such's Xaml, though obviously different teams at Microsoft may have different opinions. DotNet's Xaml is without a doubt more powerful and flexible, but as with anything with those descriptors that likely comes with drawbacks that are the reason UWP is so scaled back. And by non purely managed, I'm refering to the one nice thing about UWP's Xaml in that it's not DotNet exclusive. So a potential future Microsoft.Xaml could be a cross-plat native implementation with a managed facade so it could be used outside DotNet. (That would also exclude it's inclusion in dotnet/corefx unless this future library was decided to ship as a component of CoreFX)

[Mike-E-angelo]

Alright, I am confused. When you say .NET... you're talking 4.6, right? From my understanding this is for a .NET Core Xaml library, which by its very nature is cross-platform and .NET (4.6) exclusive? When you say "not DotNet exclusive" you mean "is DotNet inclusive"? Double-negatives. :stuck_out_tongue: This is obviously not true because you cannot use UWP Xaml in .NET 4.6 (nor would you want to :stuck_out_tongue:).

Please enlighten me on what I have wrong here. Additionally, a good goal here would be to get Xaml shipped as part of CoreFX. Some discussion has been going around this for some time. -- and sure enough, if I am not mistaken @RichiCoder1 you have your vote to do exactly this in that thread... I was wondering where I have seen your tag before! :smile:

[Mike-E-angelo]

And I would also like to say that I have also heard the internal leanings towards UWP's Xaml, but that was before there was some community discussion around it starting in that thread above and beyond (ala Tim Heuer's Twitter discussion). The needle has definitely moved since then. At least, it had better! LOL. There are those with the misconception that "Xaml is for UI" and then there is the rest of us that understand how it works. :stuck_out_tongue:

[RichiCoder1]

When I say DotNet, I mean DotNet including core and framework. And I realize you can't use UWP Xaml outside UWP apps, I was referring to the ability to use it in C++ UWP apps. So when I say not having it be DotNet exclusive, I mean being about to use it from C++ (or potentially other languages). And I imagine the reason why the team that owns XAML doesn't consider use cases outside UI seriously is because it's either uncommon or, in some cases, actively maligned with Microsoft or with partners. Not my personal opinion, I appreciate it's expressiveness and power.

[Mike-E-angelo]

OK great @RichiCoder1 thank you for your explanation. I think I almost completely follow you now... when you say "framework" you mean 4.6?

In my view, I will say that part of the reason why the internal team feels that it's uncommon to use it outside of UI has to do with poor engagement with developers, as I think Tim's post shows. I am also guilty of this as I really didn't get chirpy about engaging MSFT until about a year or so ago? In fact Tim's Tweet was about a vote that I had made, after I had been personally/privately complaining to everyone since Windows 8.0 that the Xaml system was completely different and unlike what it should be. When I heard rumblings that UWP (Windows 10) Xaml was basically the same -- after 3 years of no innovation or improvement -- that's when I stepped up. In hindsight it should have been much sooner.

Safe to say that was a major learning lesson. I am (or at least feel like I am) now much more integrated in the ecosystem and feel like I am being heard. In fact, a very important vote I made a little over four months ago was marked as Under Review by the prestigious Visual Studio team earlier this week. To say my world has been rocked is an understatement. :smile:

[RichiCoder1]

By framework, I'm using it to refer to desktop .Net. That seems to be the popular way to differentiate between Core and the Full Framework.

I agree. One of the nice things about all of Microsoft recents moves is it almost forces better engagement with developers :).

Very excited about that myself! Now just to see how that plays would. That could mean anything from making the Xamarin tooling first class aside UWP apps using their shared projects, or it could mean something high level then even that. Only time (and I guess, //Build) will tell.

[Mike-E-angelo]

OK cool... thank you for the info. :+1:

As for //build... indeed. :) Personally, I would like to see a new model altogether that is (loosely?) based on UWP and utilize the new Xamarin goodness to move it to iOS/Droid. Stronger Xaml model wouldn't hurt, either. Guess we should keep this thread quiet until after April 28th, after all. :stuck_out_tongue:

[maxima120]

I hope you guys can do it ! :)

[birbilis]

Guess the UWP guys would more easily adopt a XAML implementation that doesn't pull-in .NET code (even if that code was .NET Core stuff).

Like a self-contained black box (or only depending on UWP, but we don't want that since UWP is non-crossplatform [yet] and if it was one would still miss a managed version of it that could run in .NET/Silverlight [can't lock your apps to just Win10 target]).

But this is more or less utopic I think, unless you make it with an abstraction layer and pluggables / drivers for the different applications of XAML (for UWP UI, for Workflow, for WPF/.NET, for Silverlight etc.). Like how DirectX seems a machine via HAL (Hardware Abstraction Layer), which also opens the possibility for HEL (Hardware Emulation Layer - see mentions at http://www.developer.com/net/asp/article.php/968461/What-is-DirectX.htm). Of course if you're not careful it could also open the door to hell regarding performance, but since DirectX could pull it, I guess such architecture isn't bad

[SariDev]

+1 for porting System.Xaml over to .NET Core. I hope to see one single unified XAML dialect (.NET Xaml, UWP Xaml, Xamarin Xaml) being used on all platforms in the future. It think a .NET Core Version of System.Xaml is necessary to achieve this.

[rhyous]

+1

[ghost]

To avoid mass notification alerts, we now have a new way to +1 the individual comments on GitHub, in order to show our interest / reaction for the topic at hand:

[Mike-E-angelo]

Haha @jasonwilliams200OK I for one do not mind mass notification alerts from community members expressing their rudimentary interest in a powerful, cross-platform System.Xaml solution. :angel: Maybe other topics I would share your pain/annoyance/frustration, however. :wink:

Valid (and valuable) point, in any case. I've added my :+1: for the cause.

[terrajobst]

@jasonwilliams200OK

Thanks for calling this out!

[Mike-E-angelo]

FWIW, I just invited the Xamarin.Forms team to join the conversation here: https://bugzilla.xamarin.com/show_bug.cgi?id=26740

Or if anything, make everyone here aware of the conversation there. :smile:

[Mike-E-angelo]

Got some great discussion going in the comments of this post: https://blogs.msdn.microsoft.com/dotnet/2016/05/06/net-core-rc2-improvements-schedule-and-roadmap/

Basically JSON vs Xaml for the new project system that appears to be underway for Visual Studio (hooray!). Feel free to join in. :) So glad to see that this is getting another look.

[ghost]

@Mike-EEE, also have a look at this roadmap: https://github.com/dotnet/roslyn-project-system/blob/master/docs/Roadmap.md. The new Roslyn powered project system is going to be .. precious.

[Mike-E-angelo]

whoa... crazy that I am just hearing about this now. this must be what Scott Hunter was referring to in the aforementioned blog post in regards to more information forthcoming in a future post. Good to see. Roslyn power grows. Although, I would feel more comfortable seeing a few issues for Xaml-serialized project definitions. :wink:

[ghost]

See the readme of that repo: https://github.com/dotnet/roslyn-project-system#welcome-to-the-roslyn-c-and-visual-basic-project-system (and the link to VS MSDN blog), it seems like we would be able to replace various components using MEF, which would make it possible to insert an additional serializer to parse project definition written in XAML. Personally, I have grown to prefer JSON over XAML/XML when it is matter of describing the project properties and dependencies etc. as JSON is less noisy (well, YAML is less noisier than JSON and the lang standard supports comments unlike std-JSON, but in dotnet world, YAML and other langs with indentation-aware syntax are yet to gain traction)

[Mike-E-angelo]

Personally, I have grown to prefer JSON over XAML/XML

SECURITY! REMOVE THIS MAN!!! Ahem, I mean... :)

I'm all for taking the best of both worlds. @SuperJMN and @Grokys might speak more to this, but I believe efforts are being made in OmniXaml to make it less verbose and chatty.

My problem w/ JSON isn't w/ JSON per se, but its design approach which also was the basis of .NET configuration (which I assume was its inspiration), and that is, the object's schema is "yet another artifact" that the developer has to manage/maintain and not the class definition. That is, in Xaml programming (as you know!) the class definition is the document schema. As a result, this lends a very natural feel to the development of the document, and also helps/assist with tooling as well.

Not to mention developer sanity. :smile: Nothing infuriates me more than having to chase down some obscure schema document and figure out how to install it and get the simplest intellisense to work.

The other issue I have with JSON is again not JSON-specific, but more of the web mindset and that is naming convention. .NET configuration suffered from this too. That is, you might be defining a POCO such as:

public class MyPoco {
    public string Property {get; set;}
}

And its JSON (or app.config) would look something like (and you can school me on this if I have this wrong!):

{
 { "property": "Hello world I have inconsistent naming conventions!" }
}

Here again Xaml is aligned between class definition and its serialized counterpart, which is why I prefer it.

I do have to say that .NET Configuration was the WORST because its design yielded four artifacts for every configuration entity:

1. The element in web/app.config
2. The .xsd file that you had to somehow magically link to the web/app.config (I never got that to work!)
3. The ConfigurationElement and/or ConfigurationSection elements that took the values from app/web.config (and mapped the inconsistent camelCase to PascalCase -- oh the insanity!)
4. The POCO that ultimately got wired in from the configuration element. (WHEW!)

I have not studied JSON's project design enough to know if there is a POCO object that is being serialized with the .JSON document or if the extraneous mapping that is outlined above is occurring as well. I would love to hear if you know the answer to this.

In any case, Xaml improves upon this design by only requiring two artifacts:

1. The class file (.cs)
2. The data file (.xaml)

Finally, thanks for that link to the readme! I am once again "that guy" who does not do the very thing that file asks. :laughing:

[ghost]

if there is a POCO object that is being serialized with the .JSON document or if the extraneous mapping that is outlined above is occurring as well. I would love to hear if you know the answer to this.

I think if you consider [DataContract] and [DataMember] attributes, then the .NET framework (on BCL level) is aiding JSON and other data serailizers. However, there is no native/lowlevel JavaScript object literals <-> C# POCO transition support by compiler as JSON to JavaScript (and many other langs) or CSON to CoffeeScript. However, unlike XML with all its XPATH XQUERY XSLT ties, JSON has the concept of schema to validate the data and JSON.net supports it like a champ.

For C#6 and above, take a look at these issues/proposals https://github.com/dotnet/roslyn/issues?utf8=%E2%9C%93&q=is%3Aopen+is%3Aissue+author%3Agafter+dictionary, especially https://github.com/dotnet/roslyn/issues/6949 and follow the links within. With the new C#6 dictionary initializer syntax and these proposal, it seems like C# is semantically becoming more JSON friendly.

I do have to say that .NET Configuration was the WORST

Agree, you may also enjoy the "Configuration" and "IOption" pattern introduced by ASP.NET team as a plug and play package: https://github.com/aspnet/Configuration, https://docs.asp.net/en/latest/fundamentals/configuration.html, which supersedes the old notorious XML-bound configuration manager.

[Mike-E-angelo]

JSON has the concept of schema to validate the data and JSON.net supports it like a champ.

Sounds like a class definition as schema. :) :) :)

it seems like C# is semantically becoming more JSON friendly.

Haha well to be sure, any object is already JSON-friendly. JSON is great for doing what it is intended to do: notate object definitions in a concise, human-readable way. And there is no doubt that JSON.net does a great job of meeting this objective. When you get into describing applications then we start getting into the area where this conversation starts to get, um, involved for lack of a better word. :smile:

In the end, there are two scenarios here (to me, at least): describing objects that are meant to be transferred between service boundaries (lightweight) and describing the applications that create those objects (significant). I personally like to strive for consistency in every way possible and minimize context-switching. As long as we keep our options (IOptions? :laughing:) open and allow developers to choose the format that makes the most sense to them (and allows them to use the tooling that makes them feel the most efficient!) then we all win here. :+1:

[mfe-]

UWP XAML has fewer features in comparison to the classic .NET WPF XAML. One of the features I miss most are TypeConverters.

I tried to create my own TypeConverter as suggested here https://msdn.microsoft.com/en-us/library/bb546926(v=vs.90).aspx (the article is for WPF TypeConverters)

In UWP I get with this “Unknown type 'car' in XML namespace 'using:App2TypeConverters'”

        <ListBox>
            <local:car></local:car>
        </ListBox>

Is it really not possible to add a custom TypeConverter in UWP? Or is there a workaround? I read somewhere that the XamlCompiler is using the internal IServiceProvider to register the TypeConverter. Maybe it can be registered with it somehow ? I'm also interested in learning more about the background regarding XAML compilation. Will XAML still be generated to BAML? Can we somehow hook into the XAML compilation? So, please open System.Xaml.

[Mike-E-angelo]

Sooooooo... does this mean that System.Xaml is finally going to join the party??? :smile: :smile: :smile: https://blogs.msdn.microsoft.com/dotnet/2016/05/27/making-it-easier-to-port-to-net-core/

[ethanhs]

I'm going to pipe in here and say: a) I really would love XAML running with .Net Core and b) ifwhen that happens, it would make the most sense to use the UWP flavor, as it is already being ported to other architectures (eg the IoT push on the Raspberry Pi).

Getting WPF XAML would be amazing though, if that is possible.

[JackUkleja]

In the same way that .NET core was essentially rebuilt from the ground up, based on the past the decade of learning, I think the same should be done for XAML. The evolution WPF/Silverlight/WinRt-XAML has been a complete debacle. One step forward, two back (one sideways). The promise of XAML, Expression Blend etc has never come to fruition.

I heard Scott Hunter justifying no cross-platform UI in Dotnet Core the other day by say "Well, they never turn out good. On Windows users expect a Windows looking button, on Mac a Mac button". Well I'm sorry - where has he been for the last 5 years??? The web entirely dominates UI design now and on the web there is no such thing as a "standard button" - but guess what? Users have managed to work it out. That argument simply stands against all evidence. These Web UIs now dominate commerce on the planet. But apparently that's not good enough for Dotnet Core. Why can't we have a cross platform XAML without pandering to any [imaginary] "standard platform look & feel" aspiration?

The main technical obstacles I presume would be how to interface "XAML" (Or whatever the next generation UI platform is called) with a cross platform 3D API - as we obviously cannot take a dependency on Direct3D/Direct2D on MacOS etc. But I presume this problem has been solved by Unity etc who manage to achieve that goal.

There has been so much foot dragging by Microsoft on the subject of open sourcing the XAML-family of technologies, one has to wonder if hidden within it is a secret cache of bitcoins or the private keys to microsoft.com.

Get on with it MSFT : Open Source WPF/Silverlight/WinRT-XAML and let the next golden era of UX begin.