search

dotnet / vblang

The home for design of the Visual Basic .NET programming language and runtime library.
290 stars 63 forks source link

Pattern-Based XML Literals #483

Open AnthonyDGreen opened 4 years ago

AnthonyDGreen commented 4 years ago

This proposal is a follow-up to the idea introduced in #110 and in this blog post. I got the idea from Lucian, who got the idea from Jonathan, who got the idea from a VB user whose name is lost to history. Could also be part of a solution addressing #397.

VB.NET's existing XML literal syntax was added in VB2008 to provide a more productive, declarative syntax for building XML documents and fragments using the XLinq (System.Xml.Linq) API object model; the goal of the proposal is to generalize this syntax to be useful for building documents, fragments, and programs in other XML-based and XML-like DSLs using other object models. Rather than being tightly coupled to the XLinq object model specifically, or any fixed set of object models known by the compiler in advance, the intent is to introduce a level of indirection in the interpretation of an XML literal as well as a an abstraction sufficiently expressive that an open-ended set of APIs/DSLs could be used from the language without further modification of the compiler. This is consistent with the patterns used for LINQ and elsewhere in the language.

Example 1.

MainPage = <ContentPage Title="My First Android App!" BindingContext=<%= Model %>>
             <StackLayout VerticalOptions="Center">
               <Label 
                 HorizontalTextAlignment="Center" 
                 FontSize="Medium" 
                 Text="Welcome to Xamarin.Forms with Visual Basic .NET!"
               />
               <Entry Text="{Binding Name}"/>
               <Button Text="Next" Clicked="NextButton_Click"/>
             </StackLayout>
           </ContentPage>

In this example (taken from a prototype demo linked below), an XML literal fragment is being used in a VB code file (not a XAML file) to initialize the MainPage of a mobile application's navigation stack. Note that with the exception of the embedded expression, the expression otherwise uses the same XAML syntax (and XML dialect) that is used in XAML documents. Rather than produce an XElement, this literal produces a strongly-typed ContentPage object initialized correctly with the expected descendants. This is done without run-time parsing and can benefit further from compile-time type checking. Further, the compiler has no special knowledge of the object model being targeted or intricacies of XAML such as type converters, markup extensions, or data-binding, yet achieves the same results.

Table of Contents

I. Prototype, Motivating Examples, High-Level Design Description

To illustrate some potential applications I have prototyped an implementation of an straw-man design of an XML Pattern. I thoroughly expect that the final shape of any XML Pattern will and should change significantly from this initial draft (in ways that are elaborated on in the later sections of this document). However, the mechanism of the draft are sound and sufficiently broad to support realistic motivational examples. I have constructed 3 such examples and recorded a series of videos demonstrating them (each 5-6 minutes in length) and a companion blog post per video further elaborating on the demo. They are as follows:

NOTE: These examples use mockups for illustration. The actual design details of any real implementations or libraries are beyond the scope of this proposal. This is not the place to raise or debate concerns about the correct way to implement an ASP.NET View engine or whether one should be created at all. There will be a time and a place for that if the language has the capabilities to leverage such libraries. While I appreciate (and indeed intend) enthusiasm for these possibilities I ask that any feedback or ideas in this area be held until that time.

Code for the Prototype and Demos

The Xamarin.Forms example was built and recorded using code from branch prototypes/vb-pattern-based-xml in my fork of the Roslyn repo.

The ASP.NET Core Web Controls and Client-Side VB.NET in Browser examples were built and recorded using code from branch prototypes/vb-pattern-based-xml-top-level in my fork of the Roslyn repo which includes commits from the "Top-Level Code" prototype introduced in issue #446. That proposal is not strictly necessary for any of the examples but does synergize well with it.

The demo code is on GitHub at AnthonyDGreen/vblang-prototype-demos in the pattern-based-xml folder.

NOTE: The code for all demos is in that folder even though the ASP.NET demos require a different branch than the Xamarin demo!

Further potential examples

Though I chose to record examples using HTML and XAML, as they are likely familiar and resonate with a broad set of viewers, one can easily imagine others such as:

Happy to hear ideas from others! Want to consider a broad set of potential applications.

High-level description of the pattern

Fundamentally these 6 points describe the what of the feature. How the compiler executes them can (and must eventually) be described in excruciating detail. But for the purpose of feedback (and the prototype) the translation is much simpler. A key element of the current implementation is a series of fallbacks. Meaning, the pattern can be satisfied with some more advanced helpers but in their absence will fallback to a simple convention that allows a lot of existing library code to be used without modification or extension.

  1. Some mechanism tells the language to interpret an XML literal expression differently than it does today.
' Prototype only.
Imports <xmlns="clr-namespace:System.Windows">
  1. XML elements with simple names create objects.
' TypeOf w Is Window.
Dim w = <Window/>

Right now this causes the compiler to call the constructor of a type named Window (case-insensitive). The type must exist and there's no indirection for a factory method or for the type to not exist at all.

  1. XML attributes and XML elements with dotted names initialize properties, events, or invoke helper methods.
' b.Content is an Image object.
Dim b = <Button>
            <Button.Content>
                <Image .../>
            </>
        </>
  1. Quoted XML attribute values may be interpreted a number of way based on the target type of the property or event being initialized or the arguments of the helper method being called.
' w.Title = "Untitled", w.Width = 1920, w.Height = 1080.
' w.Loaded event will call LoadedHandler in containing scope when raised
Dim w = <Window Title="Untitled" Width="1920" Height="1080" Loaded="LoadedHandler"/>
  1. Child XML elements which create objects are added to the objects created by their container via helper methods.
' w.Content is a Grid object.
Dim w = <Window>
            <Grid/>
        </>
  1. Embedded expressions may appear as XML attribute values or child elements and are interpreted as normal VB expressions.

Translation details

The prototype demostrates that a fairly straight-forward translation is possible but the actual details used in the prototype are minutiae at this time. Eventually they'll need to be written out but they would massively bloat this issue to detail inline. I've uploaded an example translation as a gist: What you write | What the compiler generates.

Note that for the XAML example you do need separate helper methods per attribute/property to avoid the compiler needing to understand any relation between the Control.Text property and the Control.TextProperty field which is required for data-binding, dynamic resources, etc.

II. Background and Discussion of XML vs Alternatives

Why XML?

The simplest answer is that XML is already in the language. At the moment it's limited and this generalization means getting far more utility out of what we already have.

Inter-operability

The next interesting answer is, in my belief, the same answer that motivates the use of XML in XAML; inter-op. Originally XAML as used by WPF was beneficial compared to the VB.NET (or C#) based code-behind imperative code used by WinForms because XML was a language of inter-operability between the tooling in Visual Studio and Expression Blend. That same inter-operability applies both because tooling can more easily understand XML than VB and because existing dialects read by humans already use XML or XML-like languages (such as XAML in Xamarin or HTML in ASP.NET Core).

I'll give two hypothetical examples of this.

Example 1. In (VS) editor rendered MathML Imagine being able to embed a fragment of MathML--an ISO standard markup language for describing mathematical notations in a VB program for some intended execution. Perhaps taking the output of tooling used by some financial analysts, mathematicians, or scientific modelers. By leaving the formulae in MathML a Visual Studio extension can render it in editor using the same rendering technology used elsewhere and the otherwise cumbersome syntax can be presented in a manner that's most readable to the developer and subject matter experts while still having a living executable artifact which can potentially be extended with code.

Example 2. Polyglot markup Combined with the top-level code feature it's very easy to imagine a file which contains only polyglot markup--a file which is both valid as VB and valid as some other markup file such as XAML. That means the same code could be valid in a VB project and still worked on using some 3rd party tooling so potentially the previewing of a .vbxaml file could be handled by the Xamarin, WPF, or some generic HTML previewer tooling but the run-time support could be pure VB.

Advantages over XML literals as they exist today

The key advantages being pattern-based would confer beyond what XML literals can do today are:

  1. Producing objects of types other than those in the XLinq API, e.g. XDocument, XElement, XAttribute, etc.

  2. Enabling the compiler and tooling to make use of richer type information than that provided by the XLinq API and to persist that type information at run-time.

The Xamarin.Forms and Web Controls examples listed above highlight the benefit of #1 straight-forwardly. They construct objects that are then consumed by other systems without first dehydrating to XLinq objects and then re-parsing and re-processing the result.

To elaborate on #2, if I write this code today:

<html>
  <body>
    <p><%= $"Including some {userInput} in my markup." %></p>
  </body>
</html>

The XLinq API affords no way to tell the compiler that the target type for the content of the <p> element is FormattableString and so the interpolated string won't be realized as such. Further, even if one were to manually instantiate a FormattableString using an explicit cast the XLinq API can't hold on to an instance of that type. Instead the FormattableString is necessarily flattened to String type during object construction so if some processor wanted to apply HTML escaping to the userInput to avoid script injection, for example, it cannot do so because that information is lost upon construction of the object graph.

The string interpolation example highlights a security benefit while the Client-Side VB.NET Running in the Browser example listed above highlights a functional scenario. Because the compiler has access to type information from the object-model being created it can apply an expression-tree transformation to a lambda expression and that representation is persistent at run-time thus enabling a later processor to convert the expression tree to JavaScript. As they are today XML literals would simply include the mangled name of an anonymous delegate type.

Are these XAML Literals?

No. The design challenge is to produce a pattern flexible enough to create a subset of XAML programs using the same syntax but without introducing the vast majority of XAML mapping concepts into the VB language. This is beneficial both for the simplicity of the VB language but also to ensure flexibility to address non-XAML DSL dialects such as XHTML, MathML, X3D, etc. How concepts such as type converters, data-binding, markup extensions, templating, etc are enabled without introduction to the language is delegated to the implementation of helper methods. Some hypothetical mockups showing type converters and data-binding can be examined here in the demo code.

Having said that, upon full consideration of the set of opportunities presented by this feature and design pain-points it may be the case that it is simplest to adopt at least some aspects of XAML directly (e.g. the XAML namespace itself, e.g. x:Name, x:TypeArguments). The goal should be to reduce this set to the smallest size possible before considering taking such a leap, if at all.

Comparison to Object Initializers

It's a logical question to ask if object initializers would be well- or better suited to this task. Languages like F#, Swift (e.g. Swift UI), and Kotlin (e.g. Type-Safe Builders) all use a syntax that's more... native to their language to achieve some declarative simplicity in expression. VB has the advantage of having an actual declarative language embedded within in but there are other reasons I feel relying on object initializers falls short.

In short, you can do some extensive and unnatural things to object-initializers to get to a place which can functionally handle the scenarios in an unsatisfying way that abstracts away none of the imperative complexities of the domain or the underlying object model, doesn't enable easy interop with other tools, samples, or documentation, and specifically for domains such as XHTML and XAML looses all resemblance to the target domain. XML is naturally declarative and, via XAML, has already been shown to be capable of expressing the needed idioms cleanly.

For a detailed analysis of these points see the V. Expanded Discussion on Object Initializers as an Alternative at the end of this document.

III. Priority Areas for Design Feedback

If I were out in Redmond today, visiting with the VB LDM, these are the topics I'd want feedback on immediately; this is how I'd spend my precious 1-2 hours of LDM-time. They're the top-of-mind issues I feel need community feedback and LDM guidance sooner rather than later.

A. What makes an XML literal pattern-based?

Here are some ideas and some of their pros/cons. It's not exhaustive and I'd love better suggestions/feedback.

1. Magic or Extended XML namespace imports

For speed and ease of implementation the prototype uses a magic namespace import inspired by XAML:

Imports <xmlns="clr-namespace:System.Windows">

What I like about this is that it means some XAML fragments might just work correctly automatically, particularly any user-mapped namespaces because we'd be re-using the same syntax. It doesn't actually work for the "built in" namespaces in WPF for example because they use a more complex mapping. In a WPF .xaml file you'll see this default namespace declaration xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" and it's a separate mechanism in the designer that uses CLR attribution and that knows to map that XML namespace to a particular assembly and several CLR namespaces--the single namespace mapping brings both System.Windows and System.Windows.Controls into scope. Perhaps a similar file- or project-level mapping mechanism could be added:

' Hypothetical syntax
Imports <xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" clr-namespaces="System.Windows,System.Windows.Controls">

This would help avoid silently changing the meaning of any existing XML literals purely because they include a recognized XML namespace.

It also makes it pretty straightforward to have separate "builders" or factories for particular XML literals by keying the builder to the namespace. So if I want to have an outer literal that's in HTML that contains an inner literal in MathML the XML namespaces will disambiguate which builder object is invoked for each.

Another thing I like about this is that if VB gets the method-scoped Imports directives this design will synergize well with that.

My problem with this design is that there's the potential for duplication in that you need a traditional VB Import statement (or project-level import) to refer to types with simple names in VB code and potentially a duplicate one to use those same types declaratively in XML:

' This will get tedious.
Imports System.Windows, 
        System.Windows.Controls
Imports <xmlns="clr-namespace:System.Windows">
Imports <xmlns="clr-namespace:System.Windows.Controls">

I wish there was a way to share that but I'm also not 100% sure on the idea that every type in scope in regular VB code should be available in an XML expression (or vice versa) as the vast majority of them won't be part of the target domain. Is there some some compromise?

ADG: For what it's worth the prototype doesn't actually bind the name specified by the clr-namespace: syntax. That got to be too tedious for manual and automated testing so the prototype just looks for that syntax to turn the feature on then just binds all type names using the regular VB Imports in scope already.

2. Extended XML document header

Right now VB only accepts XML document headers like this: 

<?xml version="1.0" encoding="UTF-8" standalone="no"  ?>

The version must be 1.0 and is required, the encoding is optional, the standalone attribute is also optional but may only be "yes" or "no" if specified. Anything else won't compile.

We could extend this with some new attribute which previously didn't compile and has some meaning or perhaps a different value that wasn't legal before. The problem with this is that if you imagine a world where there are top-level only VB XML literals like in the second and third prototype example videos, those top-level XML docs may no longer be polyglot markup. That is to say, other XML tools might reject them is we deviate from the structure of well-formed XML.

The other problem with this is that it requires this document declaration on every outer-most element that appears in a file; the feature would only apply to pattern-based XML documents and not any XML literal.

3. DTDs (!DOCTYPE)

VB XML literals today don't support specifying a DTD. I think co-opting the !DOCTYPE declaration has similar pros and cons to extending the XML document header + the con of introducing even limited awareness of the concept of !DOCTYPE into the language.

The 2008 VB Team chose not to cross this bridge. I'm inclined not to do so either but listing this for fair discussion.

4. Magic XML processing instructions

XML actually supports the idea of an open-ended capability for special instructions to the tool processing it: XML processing instructions.

Dim x = <?xml version="1.0"?>
        <?vb?>
        <document>
        </document>

VB supports including such instructions today. Almost anything can go between the two question marks. It's straightforward enough to reserve some XML processing instruction target for the VB compiler itself. Might need to twiddle with parsing a bit to let you specify them before standalone elements so as not to require a document header but the approach seems inline with the intent of the design of XML.

5. Target-typing

One of the tried and true methods of making an expression do one thing sometimes and another better thing other times is target-typing. That's the idea that when an expression is in a context where it's being converted to a well-known type the compiler handles it specially. This mechanism is how array literals, interpolated strings, lambda expressions, and others handle their magic. The idea is that given this statement:

Dim control As TextBox = <TextBox Text="Blank" />

That the conversion from XML Element expression to TextBox is what triggers the pattern-based processing. There are a few problems with this approach.

First, that depending on the scenario a target type won't be available. And it's very easy for target-type context to be lost, for example when using the If operator.

Second, the target type will likely be less derived than the type being instanced. Change the above example to:

Dim control As Control = <TextBox Text="Blank" />

How does the compiler know to look for TextBox based on a target type of Control? Does it look in the same namespace?

It's not impossible to use this mechanism but you have to look very carefully at common usage scenarios to see if it falls apart too often.

B. What does the actual construction of an object created by a pattern-based XML literal?

Currently the prototype uses a combination of methods 1 and 2 below though I think a combination of 1 and 3 might be preferable.

1. The Compiler, Based on Convention

In this model the compiler maps XML idioms to CLR types and member operations based on conventions, a la XAML. So:

This approach is the simplest and puts the least burden on implementers because with the right set of conventions most APIs (i.e. WPF and Xamarin.Forms) will work out of the box for simple idioms. The compiler can have a little bit of smarts about converting attribute values to primitive types or enum members but not much else. For a simple DSL this may be sufficient but for more interesting DSLs (e.g. XAML) some set of helper methods will be required. The danger is that not every object model will fit the convention already or even be able to.

A downside to convention is just how many "conventions" you have to build into the language. So perhaps the 3 above seem reasonable but what about things like Attached Properties & Events? It's certainly possible to say <Type X.Y="Value" /> maps to Dim temp = New Type : X.SetY(New Type, Value) but it feels bizarely XAML-specific (though the prototype does implement this convention currently for testing). We can totally decide to live with that but it's a thing to be aware of. For what it's worth, realistically all of the scenarios which require attached properties would also require advanced helper methods for type converters and data-binding so supporting the convention really just saved me time for my demos; it wouldn't be enough in the real world.

This approach is "stateless" so if there's any need to "remember" anything there's no mechanism for that (more on that below).

2. Instance/Extension Methods

In this model the compiler maps XML idioms to instance or extension methods on objects being constructed. So:

This approach gives more flexibility because an object-model can be retrofitted to support the XML idioms after the fact in the same way that LINQ operators can be added to any type via extension methods.

My main problem with this is that all of these helper methods clutter the declaration space on the types with confusing, irrelevant, and almost always duplicate members. So in addition to having a View.LayoutOptions property that accepts a LayoutOptions structure there's also a View.SetLayoutOptions method (or extension method) that understands how to map data-binding for the LayoutOptions property and type conversion for the LayoutOptions structure. There's a helper method (potentially) which knows how to map adding children to an ItemsControl in addition to whatever Children.Add property/method are already available on e.g. Panel. I love LINQ but I don't love how dozens of almost always irrelevant members are added to the completion list on String or any other collection type to support it, particularly because I use query comprehensions more than lambda syntax. This is primarily a tooling problem so a tooling solution could work.

I'm not too worried about name collisions but another implicit requirement of this approach is "having an instance to call it on". You also don't have a chance to intercept object construction, i.e. no way to say <br /> maps to GetOrCreateLineBreakSingleton(). The Roslyn API, for example, in many places doesn't expose public constructors at all, instead relying on factory methods.

Like the above this model is virtually stateless--each XML "operation" is an independent method call with no real way to track a larger orchestration.

3. Builder

In this model the compiler, on either a per type or per XML namespace basis, gets or creates some kind of "builder" object and then interacts with that builder to implement the pattern idioms. This is similar to the GetEnumerator/GetAwaiter design. So

This is the most complex/work for the implementer of the pattern (not the end-developer using the pattern). It has the advantage of relegating all the infrastructure of supporting the pattern to the declaration space on the builder type and, hypothetically, the builder can still be extended. You can potentially do some exotic things with this approach (see "5. Exotic construction patterns") and the builder can be stateful (per literal expression). So potentially it could remember things like other XML namespaces in scope (useful for data and control templating, or runtime evaluation of markup extensions) or certain optimizations. There might also be some benefits in the binder if the compiler can compute/cache certain lookups on a per XML namespace basis.

4. Fallbacks

I dislike all-or-nothing designs where if one thing needs to be tweaked, a simple requirement explodes into a massive requirement. For that reason the prototype is implemented to look for helper methods but can fall-back on conventions. So for a very simple DSL just making the object model itself might be enough. But for targeting something like XAML the capability is there. Even within XAML/WPF/Xamarin, some properties are bindable and some aren't. It's convenient to not need to write anything special if things like data-binding or other markup extensions aren't needed but to be able to add them when they are needed.

5. Exotic construction patterns

Because my initial example was Xamarin XAML the prototype design naturally makes assumptions that might not apply to all domains and it's definitely worth considering whether those assumptions should be baked in. For example:

It would be great to get more feedback on the flexibility needed to support a broad set of potential DSLs.

C. Other important things to think about now

1. Open-ended XML schemas/expando-attributes (e.g. HTML)

Some DSLs enable or require the acceptance of undefined attributes or elements. There should probably be a a pattern for allowing or disallowing such, e.g. Function XmlElement(elementName As String) As GenericHtmlContent

I don't think you always want to be open ended because that removes some of the benefits of the static typing but some support for openness would probably be great.

2. XAML Member-Element syntax vs child-elements/dotted-names

The prototype always treats <X.Y> as though it's initializing a child element Y on an object of type X because that's what that means in XAML. Names with 2 or more dots are just forbidden for the same reason. But there are surely XML dialects and DSLs where member names of object/type names would naturally contain dots. I think Android's UI markup does this and would like to hear about more. Should there be a convention/fallback around this? How can we future proof this? What about other names with invalid VB identifier characters, e.g. command-name="Open"?

3. XML Misc (Comments, whitespace, processing, entities?)

4. Instancing generic types

Xamarin XAML makes use of generic objects, e.g. OnPlatform and OnIdiom and uses the x:TypeArguments attribute to facilitate this but it looks like that usage is deprecated.

End-developers can work around it pretty easily using inheritance though. I can't think of another good way to support this without special-casing the x:TypeArguments attribute. Technically in XAML this namespace is reserved for XAML processors. We could take the plunge and say that in this context VB is very much that.

Are there some other compelling scenarios for generics in a DSL markup language?

5. The x: ("http://schemas.microsoft.com/winfx/2006/xaml") namespace.

This namespace is reserved in XAML. All of my demos were done without special casing anything in it.

I'm trying hard to not special case anything in here but someone should make the opposition case for at least some of the attributes. I still think we should hold fast on markup extensions being opaque to the language. Might just need to settle for this feature supporting "most" XAML fragments and not "every conceivable" fragment.

D. Opportunities for better tooling, better scenario support, nicer experiences, or supplemental language features

1. Pseudo-declarations for XML "symbols".

In the prototype, some XML idioms are assumed to map to literal CLR declarations--elements to types while others may or may not map to a "real" CLR declaration. It may be preferred to introduce XML specific 'pseudo' declarations and first map the XML idioms to those abstractions and reduce those abstractions to implementation declarations at a later time. By analogy, given a tuple type (Width As Double, Height As Double) the Width and Height "properties" don't exist at run-time but do exist for the purpose of the language. This would let the IDE provide IntelliSense for the attributes (or even elements) even if those attributes aren't backed by CLR members but rather helper methods as well as letting us fiddle with things like F1 help or documentation. So, for example, even though the Header attribute is implemented calling the XmlAttributeHeader method, F1 help would still go to the HeaderControl.Header property.

Further, for even a simple XML dialect with no backing object-model one could potentially provide limited IntelliSense by simply implementing a builder with a set of helper methods.

2. InitializeComponent special-case

For desktop, mobile, and web, there are scenarios where the pattern is that the markup doesn't instance a new top-level object but rather initializes the type it appears in:

<Window Width="1920" Height="1080">
</Window>

In this case, you don't want to make up a new Window but initialize members of the containing Window. One incredibly hacky way to accomplish this is by calling a helper with a ByRef parameter before any other initialization which replaces the instance being initialized with the instance passed in. But that's really hacky see example.

Instead of that, I'm going to propose a much more targeted fix. If a pattern-based XML literal appears as an expression statement in an instance Sub named InitializeComponent, and the result type of the literal is compatible with the type of the instance, the compiler will apply the initializers/calls/etc of that literal to the instance. The reason this isn't as gross as it sounds are 2:

The benefit of this special case is that if combined correctly with the #446 proposal this preserves polyglot markup (a file that is both valid VB and a valid XAML file). The alternatives require some VB-specific content.

3. "Implicit-Conversion" from expression E of type T to Func(Of T) or Expression(Of Func(Of T)) for templating.

Given this:

<outer>
    <inner />
</outer>

The nested XML fragment is processed as an expression. It's as if it were written <%= <inner /> %>. I'm proposing that the declarative syntax support an implicit lambda conversion so that if the helper method is typed as such this code can be interpretted as <%= Function() <inner /> %>. This is how data-templating would logically be handled in the .NET type system and in fact the Xamarin.Forms object model uses a delegate for data templates rather than... whatever unholy things WPF does for that reason. Obviously because VB supports embedded expressions you could express the lambda explicitly but it would be nice to in this context support the re-interpretation declaratively.

Of course, this should also support converting to Expression(Of Func(Of T)) because then your templates can be examined as expression trees. Then in the ASP.NET scenario you could write a template in VB XML that's translated into the javascript producing that DOM on the client side. See prototype demo 3 - client side VB for a related example of the power of this feature in combination with expression tree conversions.

I haven't prototyped any of this as of yet but I think it's really cool and really powerful and so I'm mentioning it now.

I'm also interested in supporting this conversion for embedded expressions as attribute values for similar reasons. e.g. <web:ListBox DisplayMember=<%= context.FullName %> />. Basically, a little more work on the translation side a lot more declarativeness for end-developers. It gives the pattern-author the power to declare whether expressions are evaluated immediately or deferred without pushing that complexity on the end-developer.

4. Expression Functions/ExpressionTreeOf operator/Handles improvements?

This is well outside the scope of this feature. It's not even like secondary to it. But in looking at the scenarios touched on by the "Client-Side VB.NET in Browser" scenario I considered some alternatives to what was shown. What I showed was a lambda expression in the markup. Here are some alternatives I just want people to stew in.

Three different approaches to providing an object model with an expression tree representing the body of a method to enable more scenarios like what I showed in the client-side video. It's not required for this proposal but was inspired by it so I thought I'd bring it up.

IV. Design Minutiae

This is the long-tail of almost trivial questions that came up during prototyping and will eventually need to be answered. They're not necessarily hard choices, the LDM just needs to run through them eventually. Some of them are just dev notes. I was originally just going to brain-dump them there to ensure I don't forget them but for fear of bikeshedding I've extracted them into a gist for that purpose. I'll do my best not to look at them for a while and as items become more clearly defined or less trivial I may migrate some of them here.

V. Expanded Discussion on Object Initializers as an Alternative

Object initializers don't, by design, reduce any complexity. They allow you to express things in the language as expressions but don't make those expressions or patterns any less imperative. While nothing in the XML pattern approach precludes extending object initializer syntax in scenarios where doing so would be natural, it would require unnatural extensions to even begin to approach the simplicity already present in XML.

Some examples:

Doesn't abstract away "type converters"

Compare these lines:

XML:

  FontSize="Medium"

Object Member Initializers:

  .FontSize = Device.GetNamedSize(NamedSize.Medium, GetType(Label)),

The initializer doesn't abstract away the implementation detail of how to get from the descriptor "Medium" to the double value for the font-size. One still has to be aware of and interact with the Device class, the GetNamedSize method, the NamedSize enumeration, etc.

Doesn't abstract away subtleties in the object models

The "How" not "What" flexibility of declarative coding means the developer is free to specify that an element is the child of some other element without specifying the particularities of the object model. Compare this example using XML:

MainPage = <ContentPage Title="My First Android App!" BindingContext=<%= Model %>>
             <StackLayout VerticalOptions="Center">
               <Label

to this one using object initializers:

MainPage = New ContentPage With { .Title = "My First Android App!", .BindingContext = Model,
                 .Content = New StackLayout With { .VerticalOptions = VerticalOptions.Center,
                   .Children From {
                     New Label With {

Note how initializers still require one to specify the exact content property of ContentPage (which won't always be named Content) and to also specify that the content of a StackLayout be added to the Children collection (which might be called Items elsewhere).

Doesn't handle events

Currently there's no way to subscribe to an event in an object initializer. That could be worked around but even a hypothetical syntax would retain some amount of ceremony:

  New Button With { .Text = "Next", AddHandler .Clicked, AddressOf NextButton_Click }

Compared to XML:

  <Button Text="Next" Clicked="NextButton_Click"/>

Yes, you could come up with various schemes to remove syntax such as New, AddHandler, and AddressOf but all of the ways of doing that introduce a lot more complexity in areas that don't benefit from it.

Can't initialize both members and elements

This is actually easier to deal with in VB than in C# because the With and From syntax disambiguate intention. But for types like panels (Grid, StackPanel, StackLayout) it's very common to need to initialize both the elements in the panel and some other aspect of the panel itself. You want to be able to express this:

  <StackLayout VerticalOptions="Center">
    <Label 
...
      />

As this:

  New StackLayout With { 
        .VerticalOptions = VerticalOptions.Center
      }
      From { 
        New Label 
      }

The object model can work around this limitation by pushing the "collection aspect" of itself into a child property but there's no guarantee the model will support that and it pushes that complexity onto consumers.

Doesn't handle methods (e.g. required for data-binding and attached properties/events)

Data-binding completely defies the scenarios that member initializers were built for. In this example using XML: 

  <Entry Text="{Binding Name}"/>

The result isn't setting the Text property at all. Rather, the imperative translation of this is:

Dim obj = New Entry
obj.SetBinding(Entry.TextProperty, New Binding("Name"))

A method call passing in a shared field and a binding object. One could imagine extending initializers to support method calls but that still shifts the burden of managing all of that boilerplate and complexity onto the end-developer. Something like:

New Entry With { .SetBinding(Entry.TextProperty, New Binding("Name") }

And it's even more complicated by attached properties and events because rather than invoking an instance method on the object being initialized those require invoking shared methods on a completely different type. So you'd want to further generalize object initializers to support arbitrary method calls in the initializer list, which pass in the instance being constructed, which can't actually be referred to yet (ok, technically initializers run after construction so it's fair game, I guess):

  New Grid With {
        .RowDefinitions = ...
        .Children From {
           New Label With { Grid.SetRow(.Me, 0), Grid.SetRowSpan(.Me, 2) }
        } 
      }

I think this example in the Xamarin.Forms docs shows where this falls apart. The author just gives up on initializers and falls back to imperative code specifically dealing with the Grid, though they API writers have made it easier by parameterizing the Add method to take the row and column spans. You could imagine code like this:

  New Grid With { ... 
      }
      From {
        { New Label With { ... }, 0, 3, 0, 1 },
        { New Label With { ... }, 0, 1 }
      }

Summary

As stated above in Section IV., you can do a bunch of things to object initializers to support much of the functionality required but in doing so you lose all declarative elegance and simplicity for the end-developer as well as any resemblance to the target domain (e.g. HTML, XAML, MathML, etc).

VBAndCs commented 4 years ago

About Target-typing: There is an easy solution; we can use a special namespace (say: system.XML.Literals.Common) to define all supported types. It can be imported by default and referred to by xml: for example, so, we can write: Dim control = <xml:TextBox Text="Blank" /> the namespace system.XML.Literals can contain other namespaces for different technologies scuch as Xamarin, WPF, Html, … etc. Developers can extend this by adding there own namespaces. The basic namespaces should be used directly through thier known shortcuts, such as xmr:, wpf:, htm: . If one need to get rid of the suffix, he can use an Imports statement in the pages to specify that all the xml literals in this page uses this default name space, or uses xmln in xml to specify the default namespace. Any other root tag that doesn't belong to these namespaces, should be treated as usual XElement. This will maintain backward compatibility not to break any working code. So, 

Dim x = <TextBox Text="Blank" />
Dim y = <xml:TextBox Text="Blank" />

x will be an XElement, and y will be a TextBox. The root element will force the namespace on its noods without the need of the prefix.

Dim z = <wpf:Grid>
                 <TextBox Text="Blank" />
             </wpf:Grid>
VBAndCs commented 4 years ago

If this is confusing anyhow, we can use a special root for typing:

Dim x = <wpf>
                <TextBox Text="blank" />
              </wpf>

The root will resolve the namespace, and the first node will resolve the desired type.

The names of these roots will be reserved, and user defined namespaces can define their roots by some attribute or so.

AnthonyDGreen commented 4 years ago

@VBAndCs,

I'm still thinking about what you said but wanted to say thanks for being first to leave feedback!

rrvenki commented 4 years ago

@AnthonyDGreen Thanks for the article linked from #488 Pattern based programming will help us write dynamic code more easily than using reflectors, currentdomain, unloading assemblies etc. Yes this is a lot bigger thing.

  1. Please let us know if Pattern-Based XML Literals will make it to public? If so when or which release?
  2. Similarly will there be a "Pattern-Based JSON literals" at least for web programming?