search

microsoft / microsoft-ui-xaml

Windows UI Library: the latest Windows 10 native controls and Fluent styles for your applications
MIT License
6.27k stars 674 forks source link

Proposal: 3D ModelViewer Control #686

Open GeorgeS2019 opened 5 years ago

GeorgeS2019 commented 5 years ago

3D ModelViewer in WinUI

Summary

A XAML control that displays interactive 3D models. This is analogous to existing controls for other types of media (images, motion graphics, videos, etc.).

Rationale

Scope

Important Notes

GeorgeS2019 commented 5 years ago

In Build 2019 State of the Union - The Windows Presentation Platform - BRK3072 => it is stated that 3D Viewer Control is coming to WinUI 2.x

It would be great if more of the information regarding the 3D viewer control can be shared through this thread as all previous attempts to get 3D support has not arrived as a concrete moving forwards.

mdtauk commented 5 years ago

Has there been any progress on declarative XAML 3D yet, it was talked about the past few Builds

jesbis commented 5 years ago

Yes we can use this issue to track it. @sohchatt can provide the latest info

GeorgeS2019 commented 5 years ago

@sohchatt From available information, it is likely that Microsoft will choose JavaScript instead of C# to provide the 3D engine behind the opensource Windows UI control.

Surveying Youtube would reveal that many existing Unity 3D c# developers are moving to Opensource Godot.

I suggest Godot because it has ONE OF THE MOST active community support for providing high fidelity Blender export (advanced vertex/skeletal animation) to Godot 3D. All existing Javascript effort e.g. Microsoft HoloJS, will not offer the open source community this advanced export from Blender to Godot.

sohchatt commented 5 years ago

@GeorgeS2019 @mdtauk Thanks for filing the issue and providing your inputs.

We've added Windows.UI.Composition.Scenes APIs in the May 2019 SDK that allow you to create a 3D scene graph programmatically. We're also creating new open-source libraries and utilities such as SceneLoader to help with constructing this scene graph from 3D file formats such as glTF.

To make these 3D capabilities accessible via markup, let's use this issue to track the development of a XAML Control for loading 3D assets.

From available information, it is likely that Microsoft will choose JavaScript instead of C# to provide the 3D engine behind the opensource Windows UI control

@GeorgeS2019 our current plan it to use the native 3D renderer that ships with the Windows OS (version 18362) for this Xaml Control. The renderer is accessed via the Windows.UI.Composition APIs mentioned above, and allows for visual consistency with other 3D experiences in the Microsoft ecosystem, such as 3D Viewer, BabylonJS, 3D in Office, etc. I'll certainly take a look at the Godot engine and learn more about its capabilities.

If this UWP desktop app is now used in Hololens or Mixed Reality, a more 3D centric user response will be trigger to give a richer experience.

This is certainly a powerful scenario that is especially enabled by having a spatially-aware compositor/renderer, and one that we are eager to enable. We are currently evaluating the feasibility and scope for MR use-cases, but it is not part of our initial staging plans for the 3D WinUI Control. However, I'll be sure to call it out as a future goal as we make more progress on building the control.

GeorgeS2019 commented 5 years ago

@sohchatt It would be great if SceneLoaded's gITF support is able to import AND RETAIN (e.g skeletal/vertex animation) FEATURES in gITF exported from Godot

sohchatt commented 5 years ago

@GeorgeS2019 As far as the workflow is concerned, I would expect that, as long as Godot's export conforms to the glTF 2.0 spec, everything should work as you'd expect it to. This approach has the added benefit of making your 3D content portable across the tools and platforms that you're invested in, without having to be dependent on any one renderer. In addition, there is an ongoing effort from the glTF group to standardize renderers as well.

I ought to point out that the Windows.UI.Composition.Scenes APIs in the 19H1 SDK are missing some glTF features (such as skeletal animations, etc.), but it is our goal to make sure that these gaps are addressed in the next update. Would you be willing to file an issue in the SceneLoader repo requesting support for missing features so we may track your request?

sohchatt commented 5 years ago

@GeorgeS2019 I've edited your original post to be more generally applicable to 3D content in all scenarios. Do let me know if you have any questions or concerns. Note that the MR feature you requested is listed above as lower priority because the Windows.UI.Composition.Scenes APIs are not composited holographically at present, but we're evaluating this capability, as mentioned above.

GeorgeS2019 commented 5 years ago

@sohchatt This is what I hope to do with the 3D modelviewer control

https://godotengine.org/article/we-should-all-use-gltf-20-export-3d-assets-game-engines image

GeorgeS2019 commented 4 years ago

Godot receives generious grant from Microsoft! Excellent! C# will be further supported in the next Godot 3.2 version. gITF for skeletal animation import will be further improved in 3.2. Godot export to UWP uses Angle. However, to have a 3D Model Viewer UI control, we still need the work on going with #686

GeorgeS2019 commented 4 years ago

@jevansaks It would be a great idea to showcase openai/gym reinforced training in 3D in Hololens 2.0 using SciSharp/Gym.NET and godot(UWP) compiled with Angle (Hololens)

michael-hawker commented 4 years ago

Wanted to link to @JustinXinLiu's preview sample from the Insider Dev Tour: https://github.com/microsoft/InsiderDevTourDemos19/blob/master/Sessions/ui

GeorgeS2019 commented 4 years ago

@michael-hawker Please keep sharing anything related to 3D ModelViewer control for WinUI here. FYI Skeletal and vertex animation library with potential for porting to WinUI 3.0

As Hololens 2.0 is now available to independent developers, 3D WinUI pioneered in 3.0 is very important.

myokeeh commented 4 years ago

I'm really interested in this.

GeorgeS2019 commented 4 years ago

@myokeeh keep pushing awareness of this. I think Microsoft will hopefully allocate more effort into 3D for WinUI 3.0

FYI We need View3D UWP app now appears as WinUI 3.0 control :-)

Felix-Dev commented 4 years ago

You can also find the link to that demo here: https://github.com/microsoft/microsoft-ui-xaml/issues/1688

This post shows me more should be done to bring awareness to that post, which is precisely about developer resources.

myokeeh commented 4 years ago

@GeorgeS2019 does this proposal include the scope to translate/rotate separate objects relative to each other with code? Think about an assembly with moving parts and relationships.

GeorgeS2019 commented 4 years ago

@myokeeh, it is important to encourage the team (e.g. @sohchatt) to continue the effort by actively providing feedback to the SceneLoader development

Yes, we are talking of bones :-)

Noemata commented 4 years ago

The WPF 3D Xaml capability was used in hundreds of internal and external facing engineering projects and is still in use. Most of these are corporate style initiatives that have little visibility yet are very strategic. A UWP GLTF based alternative, with Xaml like binding capability cannot come soon enough. Having bindings for a GLTF bone structure and animations would be extremely useful/powerful.

StephenLPeters commented 4 years ago

@GeorgeS2019 did you mean to close this proposal?

GeorgeS2019 commented 4 years ago

@StephenLPeters I will reopen it. Closed accidentally.

Noemata commented 3 years ago

Given this: https://github.com/microsoft/InsiderDevTourDemos19/tree/master/Sessions/ui

What's happening with SceneLoad. If this is being abandoned, please say so.

Noemata commented 3 years ago

@sohchatt , Any news?

giovanni-a commented 12 months ago

I hope this is an appropriate avenue for sharing, but given the expressed desire for expanded 3D functionality in WinUI and noting the slow progress on the 3D XAML front, I felt some might find our work relevant to the discussion.

Our team, the same behind OpenSilver, is actively developing XR# (see website and twitter), a distinct framework centered around 3D application development with .NET, C#, and XAML/MVVM. It provides a WPF-like experience for developers and facilitates 3D app development for the Web, VR/AR devices, and multiple platforms by incorporating technologies like WebXR, Three.js, and the Microsoft .NET to WASM compiler. Our primary focus is enterprise VR/AR applications with a GUI-centric approach, as we firmly believe that VR/AR has applications beyond just gaming. Also, we're soon unveiling a WYSIWYG 3D XAML designer.

I share this not as an advertisement, but in the spirit of collaboration and hope it contributes meaningfully to the ongoing conversation. If this wasn't the right forum, I apologize and appreciate the understanding.

GeorgeS2019 commented 3 months ago

OpenSilver

A WYSIWYG 3D XAML designer.

Perhaps Godot as UI Control https://github.com/migeran/libgodot_project/issues/1#issuecomment-2103623279