Open stevenbrix opened 4 years ago
The compilation process is basically split into two steps:
1) AST transformations that take a somewhat "raw" nodes that contain only text info extracted from XML and produce an AST tree with resolved properties, calls, etc that can be converted into code
2) Conversion of said AST to MSIL. Currently some nodes emit themselves (usually simple ones like one responsible for {x:Null}
while others have separate "emitters" like PropertyAssignmentEmitter since the IL generation logic was too huge to fit into rather small AST node itself .
I'm not really familiar with type metadata that's available for C++, but if it has types, properties, methods and enums in a form that is easily accessible from C# code, we could make a type system backend based on said metadata and convert the resulting AST into C++ code which could be feed to C++ compiler and won't require any runtime parser support.
Not sure how compatible it would be with BAML. XamlIl essentially takes a XAML document and converts it into AST that consists of imperative commands like "create an instance of type Foo", "set the property of created type to X", etc. Some AST nodes represent local variables.
There are also somewhat hard-coded AST transformations that expect list/dictionary types to support IList<T>
and IDictionary<T>
, but transformations are defined in easily configurable way, so compiler customizations like AvaloniaXamlIlCompiler (separate repo) can add their own or replace existing ones.
I'm not really familiar with type metadata that's available for C++, but if it has types, properties, methods and enums in a form that is easily accessible from C# code, we could make a type system backend based
Yeah, this is essentially what we do today for UWP markup compilation. For xlang this sort of thing does exist in the form of metadata file. I forget the extension, it used to be .winmd
back when this was WinRT only. It's the same format as a .net reference assembly, with a special bit in the metadata noting that it's for xlang. We generate C++ code that hooks up to the XAML runtime through IXamlMetadataProvider. We do the equivalent thing for C# and VB.
Not sure how compatible it would be with BAML. XamlIl essentially takes a XAML document and converts it into AST that consists of imperative commands like "create an instance of type Foo", "set the property of created type to X", etc. Some AST nodes represent local variables.
BAML and XBF are binary representations of XAML. This sounds very similar to what we do today :)
As I mentioned before (https://github.com/AvaloniaUI/Avalonia/pull/2734#issuecomment-540664816) we want to make a more modular XAML compiler. We haven't started working on anything, so I'm very glad I reached out to you guys because I don't think I ever would've known about XamlIL otherwise :)
This is all hypothetical at this moment, but I want to start working with the community and build a more structured and unified XAML language and tooling story. Would you be open to making (or accepting) the necessary changes to XamlIl so that it fits the needs we have and help us reach this goal?
Unified XAML tooling would be great. Back in the xaml standard discussion days I've made a proposal to somewhat unify XAML platforms so they could at least use the common parser and people won't have to write trivial markup extensions multiple times.
I'd be happy to make the compiler more unified, but I'm not quite sure that XamlIl's architecture really suits the BAML/XBF outputs. They could be produced from some intermediate representation of the AST where we have everything resolved but don't have any imperative constructs yet, but that effectively removes half of the compiler from the play.
To give you a better idea of compiler's internal structures:
<UserControl xmlns="https://github.com/avaloniaui"
xmlns:pages="clr-namespace:ControlCatalog.Pages"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
x:Class="ControlCatalog.MainView">
<StackPanel>
<Button Background="Red">Hello world!</Button>
</StackPanel>
</UserControl>
initially gets parsed into
Type: XamlIlAstClrTypeReference
Type: ControlCatalog.MainView
Children: XamlIlAstObjectNode
Type: XamlIlAstXmlTypeReference
XmlNamespace: https://github.com/avaloniaui
Name: StackPanel
IsMarkupExtension: False
Children: XamlIlAstObjectNode
Type: XamlIlAstXmlTypeReference
XmlNamespace: https://github.com/avaloniaui
Name: Button
IsMarkupExtension: False
Children: XamlIlAstXamlPropertyValueNode
Property: XamlIlAstNamePropertyReference
Name: Background
DeclaringType: XamlIlAstXmlTypeReference
XmlNamespace: https://github.com/avaloniaui
Name: Button
IsMarkupExtension: False
TargetType: XamlIlAstXmlTypeReference
XmlNamespace: https://github.com/avaloniaui
Name: Button
IsMarkupExtension: False
Values: XamlIlAstTextNode
Text: Red
Children: XamlIlAstTextNode
Text: Hello world!
which gets transformed into:
Value: XamlIlAstNewClrObjectNode
Type: XamlIlAstClrTypeReference
Type: ControlCatalog.MainView
Manipulation: XamlIlManipulationGroupNode
Children: XamlIlObjectInitializationNode
Type: ControlCatalog.MainView
SkipBeginInit: False
Manipulation: XamlIlManipulationGroupNode
Children: XamlIlPropertyAssignmentNode
Property: XamlIlAstClrProperty
Name: Content
DeclaringType: Avalonia.Controls.ContentControl
PossibleSetters: 1
Values: XamlIlValueNodeWithBeginInit
Value: XamlIlAstLocalInitializationNodeEmitter
Value: XamlIlAstNewClrObjectNode
Type: XamlIlAstClrTypeReference
Type: Avalonia.Controls.StackPanel
Local: XamlIlAstCompilerLocalNode
Type: Avalonia.Controls.StackPanel
Children: XamlIlAstManipulationImperativeNode
Imperative: XamlIlAstImperativeValueManipulation
Value: XamlIlAstCompilerLocalNode
Type: Avalonia.Controls.StackPanel
Manipulation: XamlIlObjectInitializationNode
Type: Avalonia.Controls.StackPanel
SkipBeginInit: True
Manipulation: XamlIlManipulationGroupNode
Children: XamlIlPropertyAssignmentNode
Property: XamlIlAstClrProperty
Name: Children
DeclaringType: Avalonia.Controls.Panel
PossibleSetters: 3
Values: XamlIlValueNodeWithBeginInit
Value: XamlIlAstLocalInitializationNodeEmitter
Value: XamlIlAstNewClrObjectNode
Type: XamlIlAstClrTypeReference
Type: Avalonia.Controls.Button
Local: XamlIlAstCompilerLocalNode
Type: Avalonia.Controls.Button
Children: XamlIlAstManipulationImperativeNode
Imperative: XamlIlAstImperativeValueManipulation
Value: XamlIlAstCompilerLocalNode
Type: Avalonia.Controls.Button
Manipulation: XamlIlObjectInitializationNode
Type: Avalonia.Controls.Button
SkipBeginInit: True
Manipulation: XamlIlManipulationGroupNode
Children: XamlIlPropertyAssignmentNode
Property: XamlIlAvaloniaProperty
Name: Background
DeclaringType: Avalonia.Controls.Primitives.TemplatedControl
PossibleSetters: 1
Values: XamlIlAstNeedsParentStackValueNode
Value: XamlIlAstRuntimeCastNode
Type: XamlIlAstClrTypeReference
Type: Avalonia.Media.IBrush
Value: XamlIlStaticOrTargetedReturnMethodCallNode
Method: XamlIlWrappedMethod
Name: ConvertFrom
DeclaringType: Avalonia.Media.BrushConverter
ReturnType: Avalonia.Media.BrushConverter
ParametersWithThis: Avalonia.Media.BrushConverter
ParametersWithThis: System.ComponentModel.ITypeDescriptorContext
ParametersWithThis: System.Globalization.CultureInfo
ParametersWithThis: System.Object
Arguments: XamlIlAstNewClrObjectNode
Type: XamlIlAstClrTypeReference
Type: Avalonia.Media.BrushConverter
Arguments: XamlIlAstContextLocalNode
Type: XamlIlAstClrTypeReference
Type: System.ComponentModel.ITypeDescriptorContext
Arguments: XamlIlStaticOrTargetedReturnMethodCallNode
Method: XamlIlWrappedMethod
Name: get_InvariantCulture
DeclaringType: System.Globalization.CultureInfo
ReturnType: System.Globalization.CultureInfo
Arguments: XamlIlAstTextNode
Text: Red
Children: XamlIlPropertyAssignmentNode
Property: XamlIlAstClrProperty
Name: Content
DeclaringType: Avalonia.Controls.ContentControl
PossibleSetters: 1
Values: XamlIlAstTextNode
Text: Hello world!
Children: HandleRootObjectScopeNode
which closely resembles the final generated MSIL (decompiled):
var context = new CompiledAvaloniaXaml.XamlIlContext.Context<MainView>(P_0, new object[1]
{
(object)CompiledAvaloniaXaml.!AvaloniaResources.NamespaceInfo:/MainView.xaml.Singleton
}, "avares://ControlCatalog/MainView.xaml");
context.RootObject = P_1;
context.IntermediateRoot = P_1;
((ISupportInitialize)P_1).BeginInit();
context.PushParent(P_1);
StackPanel stackPanel = new StackPanel();
((ISupportInitialize)stackPanel).BeginInit();
P_1.Content = stackPanel;
context.PushParent(stackPanel);
var button = new Button();
((ISupportInitialize)button).BeginInit();
var children = stackPanel.Children;
((AvaloniaList<IControl>)children).Add(button);
context.PushParent(button);
button.Background = (IBrush)new BrushConverter().ConvertFrom(context, CultureInfo.InvariantCulture, "Red");
button.Content = "Hello world!";
context.PopParent();
((ISupportInitialize)button).EndInit();
context.PopParent();
((ISupportInitialize)stackPanel).EndInit();
context.PopParent();
((ISupportInitialize)P_1).EndInit();
StyledElement styled;
if ((styled = (P_1 as StyledElement)) != null)
{
NameScope.SetNameScope(styled, context.AvaloniaNameScope);
}
context.AvaloniaNameScope.Complete();
However if we stop the transformation process before XamlIlConvertPropertyValuesToAssignmentsTransformer
, we can get the following AST
Type: XamlIlAstClrTypeReference
Type: ControlCatalog.MainView
Children: XamlIlAstXamlPropertyValueNode
Property: XamlIlAstClrProperty
Name: Content
DeclaringType: Avalonia.Controls.ContentControl
Values: XamlIlAstObjectNode
Type: XamlIlAstClrTypeReference
Type: Avalonia.Controls.StackPanel
Children: XamlIlAstXamlPropertyValueNode
Property: XamlIlAstClrProperty
Name: Children
DeclaringType: Avalonia.Controls.Panel
Values: XamlIlAstObjectNode
Type: XamlIlAstClrTypeReference
Type: Avalonia.Controls.Button
Children: XamlIlAstXamlPropertyValueNode
Property: XamlIlAvaloniaProperty
Name: Background
DeclaringType: Avalonia.Controls.Primitives.TemplatedControl
Values: XamlIlAstTextNode
Text: Red
Children: XamlIlAstXamlPropertyValueNode
Property: XamlIlAstClrProperty
Name: Content
DeclaringType: Avalonia.Controls.ContentControl
Values: XamlIlAstTextNode
Text: Hello world!
which, I think, could be converted to BAML
You're AST looks fairly similar to what our node streams look like today, but I'm not our XBF/Compiler expert though, so I'll let @alwu-msft comment on that. We don't use an AST, which is one of the big things we've been wanting to address and fix, and a big reason why we want to redesign our tooling.
Unified XAML tooling would be great. Back in the xaml standard discussion days I've made a proposal to somewhat unify XAML platforms so they could at least use the common parser and people won't have to write trivial markup extensions multiple times.
I don't think I could've articulated it any better, what you said in that issue summarizes the issue perfectly! I want to be careful to not use any "Xaml Standard" rhetoric, even as a "Platform Standard", mostly so people don't associate what we're trying to do with that unfortunate series of events.
I'd be happy to make the compiler more unified, but I'm not quite sure that XamlIl's architecture really suits the BAML/XBF outputs. They could be produced from some intermediate representation of the AST where we have everything resolved but don't have any imperative constructs yet, but that effectively removes half of the compiler from the play.
Awesome! Yeah I think we can dive into this a bit more, it seems like a great place to start IMO :). But if you're honestly thinking it would be better to start from scratch, then that's good to know as well. I understand that switching out the backend to produce XBF/BAML does effectively remove have the compiler from play, but I think it's a necessary thing to do for v1. Otherwise, I think all the changes that we'd be asking for each of the runtimes to make would make this effort DOA. Once we have some level of convergence, I think we can then start exploring ways to converge even more.
I understand that switching out the backend to produce XBF/BAML does effectively remove have the compiler from play, but I think it's a necessary thing to do for v1
I take this back a little bit. I spoke with @jkoritzinsky today and since this doesn't require any significant runtime changes for WPF, we might not need to do this. We would need an abstraction of some sort for WinUI, since WinRT activation would be slow, even for C#
Well. Any chance for this getting integrated into the next versions of WPF or UWP @stevenbrix? Would really love to see platforms other than Avalonia using the XamlIl compiler. Compile-time checks and the ability to debug XAML code right in the IDE debugger make the development of complex user interfaces a real pleasure indeed 🚀
@worldbeater I can't say for sure, but it seems like WPF-xaml will be replaced with WinUI-xaml, which works with COM world. Same for UWP. I am not sure, if it is easily possible or convenient to use xaml complier on c# for that.
But it could be used for MAUI or UNO tho.
@worldbeater just to be clear, I want to keep this conversation going, but I'm not able to make any guarantees at the moment because we really need to focus on WinUI3.
I am personally a big fan of this idea, but there is still some work that needs to be done in order to integrate ~XamlIl~ XamlX into those platforms, such as x:Bind
, x:Load
, and x:Phase
for WinUI. With WinUI3, we just don't have the time or resources to dedicate to this, as we have other higher priorities, like making sure we have parity with existing experiences.
but it seems like WPF-xaml will be replaced with WinUI-xaml...
@maxkatz6 WinUI should hopefully be able to replace WPF. But that doesn't mean we won't make the proper investments in WPF if they have the proper cost/benefit. Primarily, anything that makes the migration from WPF->WinUI easier is an area of interest. If sharing the same core compiler tech is able to provide value in that regards, then that would make the case for moving to XamlX even more compelling.
...which works with COM world. Same for UWP. I am not sure, if it is easily possible or convenient to use xaml complier on c# for that.
There are some other considerations for WinUI as well since we use WinRT (based on COM). There shouldn't be a problem using XamlX for our .NET customers, but we would have to think about what to do for C++. In general, I think the two things we need to do are:
XamlDirect
would just go through the regular code path.This should be doable, so these aren't major blockers or anything.
@stevenbrix thanks for information!
The refactoring to separate the frontend from the IL backend is in. So it should be possible to build a C++ backend for XamlX without much issue.
Hey @kekekeks, I'm looking through the architecture of this and wondering how tightly coupled to generating IL this is? As you know, I'm trying to get an effort started around unifying xaml tooling across all platforms. Both PresentationBuildTasks (WPF) and the UWP markup compiler have tight dependencies on the platforms, so what you've done here is incredibly interesting to me :).
With WinUI, we have c++ customers to support, so we can't depend on IL. I also have a feeling we'd need a way to switch out backends so we could continue to generate BAML for WPF and XBF for UWP, IL for Avalonia, etc.
Also, and this might just be me, but all the
IXamlIL*
types are hard to read, mostly because of thelIL
letters in succession. It also makes me think that the whole thing is tightly coupled to IL, which I'm sure is not true, but it's also hard to get out of my head :)