Proposal Add 3DSurfaceView or 3DCanvas or Viewport3D

[insinfo]

The idea here is a control similar to Android GLSurfaceView, which is also similar to the idea of ​​the D3D11Image Source of WPF Image control. So with multiple sources like OpenGL/Direct3D/Vulkan/Metal, depending on the platform you can choose the render backend.

<Window x:Class="Application.MainWindow">

    <Grid>
        <3DSurfaceView >
            <3DSurfaceView.Source>
                <Direct3D11Source x:Name="InteropImage"/>
            </3DSurfaceView.Source>
        </3DSurfaceView >
    </Grid>
</Window>

[GeraudFabien]

Hello,

• Im not sure it is really "Widely used" (https://github.com/Microsoft/xaml-standard/blob/staging/docs/reviewboard.md#principles).
• Most of the time you don't use 3D.
• When you want 3D you want to optimise between plateform
• you should use a 3d engine with a xaml interface (I know there are). You also can render 3D in a custom control and display it (I have done it in silverlight).
• Too complex and Framework cannot add to it.

[birbilis]

What does most of the time mean? For which class of developers and for which class of applications?

Being able to define a scene graph in XAML that allows data-binding (and wiring together elements in the graph declaratively) would be nice, especially for AR/VR apps.

2D UI controls could still exist shown on 2D surfaces (like Hololens does) or even render and allow interaction on 3D surfaces (using some topology / coordinate remapping). However, could even have real 3D UI controls implementing the same semantics as some of the 2D controls if they're defined in a way that coordinates are abstracted behind types like Location/Position instead of X and Y.

[GeraudFabien]

What does most of the time mean?

It mean you need it in very specific application (3D game, 3D Modeling Tools, ...). I'm pretty sure 50% of the real application didn't have 3D. I just understand by reading another time the issue that it was proposing what i propose. A control that just help to interact with a real 3D engine. I'm sorry i didn't understand the thread.

[rashadrivera]

I think this is a lofty goal that does not add value to the full spectrum of applications already out there. It would be nice once the standard is fully mature in my opinion. I build WPF & UWP apps and have yet to come across a use case for this, but if I was doing more graphical or gaming apps, I would definitely want this.

Perhaps the XAML standard should accommodate namespace extensions so it can be introduced via a third party library or via GitHub as a .NET Standard library.

[michael-hawker]

I don't know if it makes sense to add an entire 3D object namespace to XAML. Most 3D applications are completely dynamic, and there are a lot of other 3D libraries out there.

I think the more important question, is how we can leverage XAML Standard alongside, on top of, or integrated with these other systems. It would be nice if you could have your initial menus and/or hud using XAML, but have just a viewport defined for the rendering surface of whatever OpenGL/DirectX/Alternate based cross-platform 3D Library your using.

[codecore]

A rendering target would enable 3D apps that are difficult today. See what Xamarin is doing with UrhoSharp. This is an enabling technology for AR applications for those of us who are not Unity fans. One should be able to choose the rendering technology (OpenTK, Vulcan, Urho, SharpDX, GLES) and maintain a full high-perf UI.

[JimSEOW]

We must have a vision of democratizing XAML-[SharpDX, OpenGLES (Angle), UrhoSharp] Interop.

The existing examples are all targeting FULL WINDOWS 3D apps, almost all for Game.

With mixed reality ecosystem supporting business Apps and the need of 3D for visualization of big data, WE NEED XAML-3D interop. The logical next step. The problem is that most 3D developers are focusing on full windows 3D game, very few on 3D business/data science applications

[caesun]

I really need the native 3D features for UWP. My project is blocked now due to lacking 3D view control like WPF has to show a surface profile. This feature is heavily used in science domain.

[JimSEOW]

There are only 3 proven choices: (1) SharpDX (DirectX) is restricted to UWP ecosystem (2) UrhoSharp is not available for W10M and W10A (Native), due to lack of support from the Urho3D C++ community and there is nothing Microsoft can do anything about it (3) OpenGL ES (Angle) is the best option so far.

There are 2 ways to make OpenGL ES to works with XAML (a) Nuget Angle (b) SkiaSharp with Angle built in.

I personally do not see WPF3D as it is a legacy approach that is limited to WPF XAML. I hope I am wrong

For developers who are serious to democratize standardized managed 3D for XAML, start using this thread to gain critical mass. Start spreading this discussion.

[JimSEOW]

XAML Standard with 3D chart.

Now we have a solution with XAML Standard with 2D Chart e.g. Microchart We need similar solution for XAML Standard with 3D chart. Most of the public open source 3D chart libraries, non commercial, are available in OpenGL, Not DirectX, Not Urho3D.

This is why OpenGL ES XAML proven solution is the way to go for XAML standard for 3D.

[JimSEOW]

We need to decide a consensus among us how to proceed. This topic is historically a fragmented situation. Most 3D developers target GAME and they care only full screen and NOT about XAML for 3D.

With UWP and Mixed Reality, we need to put more effort into XAML standard for XAML interop for 3D Display.

Start joining here to share your view.

[Syn-McJ]

I agree we need this badly. How cool would it be to take Paint3D for quick prototyping and export model form it into UWP app with couple of clicks?

[JimSEOW]

@Syn-McJ Microsoft is to big to have different parties of interest. XAML Standard for 3D is one of the KEY directions to bring more 3D contents to Mixed Reality especially to democratize View 3D or Mixed Reality Viewer to compete better with ARKit (Apple) and ARCore (Google)

Please spread this discussion, so the different fractions of Microsoft understand this need and TIMING is critical. NO MORE DELAY!!!!!

[dotMorten]

I don't really get the point of this:

        <3DSurfaceView >
            <3DSurfaceView.Source>
                <Direct3D11Source x:Name="InteropImage"/>
            </3DSurfaceView.Source>
        </3DSurfaceView >

The Direct3D11Source would be Windows only so you would need an OpenGL on for Android and iOS. Also the hosting control settings would wildly different, so the benefit of source sharing quickly goes out the window. I think developers are better off creating this object themselves on each platform, and that way get the shared XAML. ie

   <myControls:My3DViewer />

Where My3DViewer is a custom control build on each platform (we use this approach today and it works great - TBH there's almost zero code share the moment we get into the GPU DirectX/OpenGL pieces, so a common markup like above really won't help at all.

[JimSEOW]

@dotMorten There is a 2 in 1 cross platform solution using SkiaSharp. 2 features (2D and 3D) within 1 common nuget installation (e.g. SKiaSharp.View)

SkiaSharp.View NOW provides XAML interop cross plateform 2D vector/image drawing using Xamarin Forms [iOS, MacOS, Android, linux GTK+, UWP, WPF(soon)]

The same nuget installation also provide XAML interop cross platform OpenGL GPU acceleration using Microsoft Angle (OpenGL ES).

XAML Standard could define a new e.g. <3DSurfaceView> [based on a modified version of the existing ] for XAML interop cross plateform (OpenGL) 3D display using:

a modified version of

• (a) SKiaSharp.View.Forms and

• (b) SkiaSharp.View

[insinfo]

@JimSEOW I agree we need this badly. I believe the XAML standard has to expose two ways to integrate 3D inside 2D UI:

The first would be to expose a rendering surface, similar to WPF's D3D11Image Source. Example:

<Window x:Class="WpfApplication5.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        xmlns:local="clr-namespace:WpfApplication5"
        mc:Ignorable="d"
        xmlns:DXExtensions="clr-namespace:Microsoft.Wpf.Interop.DirectX;assembly=Microsoft.Wpf.Interop.DirectX"
        Title="MainWindow" Height="350" Width="525">
    <Grid>
        <Image>
            <Image.Source>
                <DXExtensions:D3D11Image  x:Name="InteropImage"/>
            </Image.Source>
        </Image>
    </Grid>
</Window>

Source: https://github.com/Microsoft/WPFDXInterop

The second would be to expose a complete 3D API similar to WPF3D. Example:

<UserControl x:Class="HostingWpfUserControlInWf.UserControl1"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    >
    <Grid>
      <!-- Place a Label control at the top of the view. -->
      <Label 
                HorizontalAlignment="Center" 
                TextBlock.TextAlignment="Center" 
                FontSize="20" 
                Foreground="Red" 
                Content="Model: Cone"/>

      <!-- Viewport3D is the rendering surface. -->
      <Viewport3D Name="myViewport" >
        <!-- Add a camera. -->
        <Viewport3D.Camera>
          <PerspectiveCamera 
                        FarPlaneDistance="20" 
                        LookDirection="0,0,1" 
                        UpDirection="0,1,0" 
                        NearPlaneDistance="1" 
                        Position="0,0,-3" 
                        FieldOfView="45" />
        </Viewport3D.Camera>
        <!-- Add models. -->
        <Viewport3D.Children>
          <ModelVisual3D>
            <ModelVisual3D.Content>
              <Model3DGroup >
                <Model3DGroup.Children>
                  <!-- Lights, MeshGeometry3D and DiffuseMaterial objects are added to the ModelVisual3D. -->
                  <DirectionalLight Color="#FFFFFFFF" Direction="3,-4,5" />
                  <!-- Define a red cone. -->
                  <GeometryModel3D>
                    <GeometryModel3D.Geometry>
                      <MeshGeometry3D 
    Positions="0.293893 -0.5 0.404509  0.475528 -0.5 0.154509  0 0.5 0  0.475528 -0.5 0.154509  0 0.5 0  0 0.5 0  0.475528 -0.5 0.154509  0.475528 -0.5 -0.154509  0 0.5 0  0.475528 -0.5 -0.154509  0 0.5 0  0 0.5 0  0.475528 -0.5 -0.154509  0.293893 -0.5 -0.404509  0 0.5 0  0.293893 -0.5 -0.404509  0 0.5 0  0 0.5 0  0.293893 -0.5 -0.404509  0 -0.5 -0.5  0 0.5 0  0 -0.5 -0.5  0 0.5 0  0 0.5 0  0 -0.5 -0.5  -0.293893 -0.5 -0.404509  0 0.5 0  -0.293893 -0.5 -0.404509  0 0.5 0  0 0.5 0  -0.293893 -0.5 -0.404509  -0.475528 -0.5 -0.154509  0 0.5 0  -0.475528 -0.5 -0.154509  0 0.5 0  0 0.5 0  -0.475528 -0.5 -0.154509  -0.475528 -0.5 0.154509  0 0.5 0  -0.475528 -0.5 0.154509  0 0.5 0  0 0.5 0  -0.475528 -0.5 0.154509  -0.293892 -0.5 0.404509  0 0.5 0  -0.293892 -0.5 0.404509  0 0.5 0  0 0.5 0  -0.293892 -0.5 0.404509  0 -0.5 0.5  0 0.5 0  0 -0.5 0.5  0 0.5 0  0 0.5 0  0 -0.5 0.5  0.293893 -0.5 0.404509  0 0.5 0  0.293893 -0.5 0.404509  0 0.5 0  0 0.5 0  " 
    Normals="0.7236065,0.4472139,0.5257313  0.2763934,0.4472138,0.8506507  0.5308242,0.4294462,0.7306172  0.2763934,0.4472138,0.8506507  0,0.4294458,0.9030925  0.5308242,0.4294462,0.7306172  0.2763934,0.4472138,0.8506507  -0.2763934,0.4472138,0.8506507  0,0.4294458,0.9030925  -0.2763934,0.4472138,0.8506507  -0.5308242,0.4294462,0.7306172  0,0.4294458,0.9030925  -0.2763934,0.4472138,0.8506507  -0.7236065,0.4472139,0.5257313  -0.5308242,0.4294462,0.7306172  -0.7236065,0.4472139,0.5257313  -0.858892,0.429446,0.279071  -0.5308242,0.4294462,0.7306172  -0.7236065,0.4472139,0.5257313  -0.8944269,0.4472139,0  -0.858892,0.429446,0.279071  -0.8944269,0.4472139,0  -0.858892,0.429446,-0.279071  -0.858892,0.429446,0.279071  -0.8944269,0.4472139,0  -0.7236065,0.4472139,-0.5257313  -0.858892,0.429446,-0.279071  -0.7236065,0.4472139,-0.5257313  -0.5308242,0.4294462,-0.7306172  -0.858892,0.429446,-0.279071  -0.7236065,0.4472139,-0.5257313  -0.2763934,0.4472138,-0.8506507  -0.5308242,0.4294462,-0.7306172  -0.2763934,0.4472138,-0.8506507  0,0.4294458,-0.9030925  -0.5308242,0.4294462,-0.7306172  -0.2763934,0.4472138,-0.8506507  0.2763934,0.4472138,-0.8506507  0,0.4294458,-0.9030925  0.2763934,0.4472138,-0.8506507  0.5308249,0.4294459,-0.7306169  0,0.4294458,-0.9030925  0.2763934,0.4472138,-0.8506507  0.7236068,0.4472141,-0.5257306  0.5308249,0.4294459,-0.7306169  0.7236068,0.4472141,-0.5257306  0.8588922,0.4294461,-0.27907  0.5308249,0.4294459,-0.7306169  0.7236068,0.4472141,-0.5257306  0.8944269,0.4472139,0  0.8588922,0.4294461,-0.27907  0.8944269,0.4472139,0  0.858892,0.429446,0.279071  0.8588922,0.4294461,-0.27907  0.8944269,0.4472139,0  0.7236065,0.4472139,0.5257313  0.858892,0.429446,0.279071  0.7236065,0.4472139,0.5257313  0.5308242,0.4294462,0.7306172  0.858892,0.429446,0.279071  "                   TriangleIndices="0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 " />
                    </GeometryModel3D.Geometry>
                    <GeometryModel3D.Material>
                      <DiffuseMaterial>
                        <DiffuseMaterial.Brush>
                          <SolidColorBrush 
                            Color="Red" 
                            Opacity="1.0"/>
                        </DiffuseMaterial.Brush>
                      </DiffuseMaterial>
                    </GeometryModel3D.Material>
                  </GeometryModel3D>
                </Model3DGroup.Children>
              </Model3DGroup>
            </ModelVisual3D.Content>
          </ModelVisual3D>
        </Viewport3D.Children>
      </Viewport3D>
    </Grid>
</UserControl>

Example 2:

<Window x:Class = "WPF3DGraphics1.MainWindow" 
   xmlns = "http://schemas.microsoft.com/winfx/2006/xaml/presentation" 
   xmlns:x = "http://schemas.microsoft.com/winfx/2006/xaml" 
   xmlns:d = "http://schemas.microsoft.com/expression/blend/2008" 
   xmlns:mc = "http://schemas.openxmlformats.org/markup-compatibility/2006" 
   xmlns:local = "clr-namespace:WPF3DGraphics1" 
   mc:Ignorable = "d" Title = "MainWindow" Height = "350" Width = "525"> 

   <Grid>

      <Viewport3D Name="viewport3D1">       
         <Viewport3D.Camera> 
            <PerspectiveCamera x:Name = "camMain" Position = "6 5 4" LookDirection = "-6 -5 -4"> 
            </PerspectiveCamera> 
         </Viewport3D.Camera>

         <ModelVisual3D> 
            <ModelVisual3D.Content> 
               <DirectionalLight x:Name = "dirLightMain" Direction = "-1,-1,-1"> 
               </DirectionalLight> 
            </ModelVisual3D.Content> 
         </ModelVisual3D>

         <ModelVisual3D x:Name = "MyModel"> 
            <ModelVisual3D.Content> 
               <GeometryModel3D>

                  <GeometryModel3D.Geometry>
                     <MeshGeometry3D x:Name = "meshMain"
                        Positions = "0 0 0  1 0 0  0 1 0  1 1 0  0 0 1  1 0 1  0 1 1  0 1 1"  
                        TriangleIndices = "2 3 1  3 1 0  7 1 3  7 5 1  6 5 7  6 4 5  6 2 0
                        2 0 4  2 7 3  2 6 7  0 1 5  0 5 4"> 
                     </MeshGeometry3D> 
                  </GeometryModel3D.Geometry> 

                  <GeometryModel3D.Material> 
                     <DiffuseMaterial x:Name = "matDiffuseMain">
                        <DiffuseMaterial.Brush> 
                           <SolidColorBrush Color = "Bisque"/> 
                        </DiffuseMaterial.Brush> 
                     </DiffuseMaterial> 
                  </GeometryModel3D.Material> 

               </GeometryModel3D>
            </ModelVisual3D.Content>

            <ModelVisual3D.Transform>                         
               <RotateTransform3D> 
                  <RotateTransform3D.Rotation> 
                     <AxisAngleRotation3D x:Name = "rotate" Axis = "1 2 1"/> 
                  </RotateTransform3D.Rotation> 
               </RotateTransform3D>
            </ModelVisual3D.Transform> 

         </ModelVisual3D> 
      </Viewport3D>

      <Slider Height = "23" HorizontalAlignment = "Left"  
         Margin = "145,271,0,0" Name = "slider1" 
         VerticalAlignment = "Top" Width = "269"  
         Maximum = "360" 
         Value = "{Binding ElementName = rotate, Path=Angle}" /> 

   </Grid> 

</Window> 

https://msdn.microsoft.com/en-us/library/ms745781%28v=vs.110%29.aspx?f=255&MSPPError=-2147217396 https://msdn.microsoft.com/en-us/library/ms748373(v=vs.100).aspx https://www.tutorialspoint.com/wpf/wpf_3d_graphics.htm

The way to handle WPF3D is fantastic, simple and democratizes 3D for everyone. See how simple it is to create and interact with 3D objects with WPF 3D.

[JimSEOW]

@insinfo I am open to MULTIPLE roads to ROME. There is ONLY ONE GOAL. XAML Standard for 3D is the platform to mobilize .NET Developers to bring more 3d content to support and grow the Microsoft Mixed Reality movement.

XAML Standard for 3D is the mean to UNIFY all 3D effort within the Microsoft/Xamarin ecosystem.

We need to SPEAK with one VOICE yet flexible to allow Innovation through Diversity (SharpDX, UrhoSharp, Angle (OpenGL ES), OpenTKCore (OpenTK).

NOW! NOW! NOW!

FOR Anyone who believe in this. Please lobby others to join in and share your opinions.

[dotMorten]

There is a 2 in 1 cross platform solution using SkiaSharp.

I know Skia very well (I helped Matt port it to UWP for Xamarin), but Skia has nothing to do with OpenGL or DirectX. Skia is a 2D graphic library for drawing/rasterizing Text, Geometries, and Images. Yes you can then use these as textures in a 3D environment afterwards, but that's again is separate from Skia.

I mean the closest you get in Skia to DirectX is that it might use DirectDraw to rasterize text, but that's not really the part of DirectX you are referring to.

SkiaSharp;.View

Did you notice that the View classes are all different on each platform? They didn't even call the classes the same thing. Yes sure you can use stuff like Angle to pretend you're coding OpenGL across all platforms, but you quite quickly get into having to really control how the gpu devides are created on each device for this to make it really useful and perform. Sure it works for these simple basic scenarios, but once you start doing more advanced rendering, it just won't do the job.

And then regarding WPF 3D: It was a horrible piece of tech that just could not perform much beyond spinning a cube around. It never caught on for 3D rendering for that reason.

[JimSEOW]

@dotMorten

DirectX will not provide the cross platform which is the aim of XAML Standard. For XAML Standard's 3D Display, I see OpenGL (OpenTK), OpenGL ES (OpenGL ES) and perhaps UrhoSharp (WPF3D is locked to Windows, not a feasible Cross platform)

Skia has nothing to do with OpenGL or DirectX

Correct: not OpenGL or DirectX . But YES for OpenGL ES

Yes sure you can use stuff like Angle to pretend you're coding OpenGL across all platforms

Yes, this discussion is about how we could tap into Angle for across all platforms

For Windows: There are github examples how to bring WPF XAML and UWP XAML to Angle For iOS, Android, Linux, The possible options are OpenTK that being ported to .NET Core 2.0: OpenTKCore Within OpenTKCore, we have Angle

In summary:

• The discussion is about exploring Angle for cross platform through the different VIEWS of SkiaSharp.VIEW.

• The discussion is about the IMPACT of such feasibility. What will 3D Display for XAML Standard have on BUSINESS 3D (e.g. Business Intelligence (BI) Visualization) instead of GAME?.

• The discussion is to highlight that OpenGL ES is more suitable for BI. There are many open source 3D chart OpenGL ES libraries/Frameworks for porting to XAML Standard's Angle 3D DIsplay. There are No open source Urho c++ 3D chart for porting to UrhoSharp.

WPF 3D: It was a horrible piece of tech

WPF 3D will locks down ONLY to Windows, not cross platform - cross platform is the motivation of XAML Standard

The first step to XMAL Standard for 3D display is JUST TO PROVIDE a 3D Planel derived from cross platform SKiaSharp.VIEW

FYI: I briefly mentioned SharpDX ONLY for situation after accomplishing Angle for XMAL Standard's 3D Display. They could be an option to allow custom rendering using SharpDX, I do agree, this is going to be very challenging.

[JimSEOW]

XAML Standard is not an independent side project, it represents Microsoft VISION. One of Microsoft's VISIONS is Mixed Reality through UWP (XAML) One of Microsoft's VISIONS is "Rich 2D/3D Understanding of Data through e.g PowerBI"

There is a clear competitive use case of combining Mixed Reality and Business Intelligence XAML Standard without supporting a 3D Display Panel IS NOT AN OPTION to realize this competitive advantage unique to Microsoft.

[bryaan]

This would be an amazing feature and would no doubt see massive adoption. Is there any plans now to do WPF3D but for UWP?

[meteorsnows]

Microsoft having great presentation like fluent, but actually moving slowly when come to customization and dynamic of UX. Especially for UWP,

Microsoft should step up the game: 1) Be the 2nd large PC Game Store...instead of Epic (Port Xbox Cloud Save to Windows 10), community upload/profile. (Revenue can be from 25%-5%, anything that Microsoft could do, in exchange for exclusivity of XBOX titles.) 2) 3D Rendering for UWP as first class citizen, also API and interop for games.

Bring communities in with ease, and build it before there is no chances.