Please provide bindings for DirectX.

[pjmlp]

Proposal: Bindings for DirectX.

Since the introduction of WinRT, later UWP, non C++ developers have suffered from lack of access to DirectX, even though they could be easily improved as Windows Runtime Components, given that they are already based on COM.

Summary

Provide the necessary metadata for Rust, C#, and JavaScript developers also to be able to integrate DirectX as Windows Runtime Components into their applications without being forced to write C++ code.

Rationale

Since XNA was put to sleep, we have been managing with community projects like Monogame, SharpDX, Vertice and many others.

They usually tend to run out of steam or fail to keep in sync why the latest DirectX improvements.

So anyone that wants to bind DirectX into their applications is eventually forced to either deal with the faults of such implementations, or having to learn C++ alongside its variants, initially C++/CX, nowadays C++/WinRT and try as much as possible to create an interoperability layer for their application.

Other DirectX related technologies like XAudio and DirectML already offer Windows Runtime Components,

There is also the Win2D project that seems to have become stale, eventually because it wasn't part of DirectX team efforts to start with.

So by having official Windows Runtime Components as part of the official APIs for DirectX, we would be assured to have up to date bindings, integrated into Visual Studio tooling, available to all Project Reunion official languages, instead of being only available to C++ developers.

Scope

Capability	Priority
This proposal will allow developers to integrate DirectX into their applications	Must
This proposal will allow developers to develop 3D games in managed languages	Could
This proposal will allow developers to use 3D in WPF, UWP and WinUI without writing C++ code	Could

Important Notes

• SharpDX (now discontinued)

Open Questions

[Nirmal4G]

There was a DirectX projection for C++ by @kennykerr during the days for Windows 8. I would like a similar effort with @microsoft/xlang.

[smourier]

Just mentioning this other community project (of my own) "DirectN": https://github.com/smourier/DirectN the difference with other projects is the whole files (more than 7000) are generated from the SDK so automatically can keep up with latest versions. It allows you to use most low level API from C# pretty easily without any runtime or projection, it's just source code. For example, I use Direct2D and DirectWrite without too much effort.

However, note that many improvements in latest Windows are not available at all even from plain C++ as "old" COM interface, only to WinRT. So I can't integrate them into that project.

For example: Direct Composition was a simple COM API that has evolved as Windows.UI.Composition (like a "dcomp v2") which is a WinRT only API (which is relatively easy to use also from plain C# with external Dispatcher Queue as explained here: https://docs.microsoft.com/en-us/windows/apps/desktop/modernize/using-the-visual-layer-with-windows-forms). The only technical way to get full integration with DWM (like for the back drop brush) is to use this Windows.UI.Composition WinRT API.

Now it's becoming Microsoft.UI.Composition I believe with WinUI3.

I wonder what these 3 levels of API are going to become with Reunion? Will we be able to use all these back again using "plain" C or C++ or even "old" .NET (with interop p/invoke like I do) without adding runtime and projection layers?

Or is the general way of adding API to Windows will still be "new Windows API" => "only to WinRT"? For example if there's a DirectX 13 or 14, will it be for WinRT only?

[pjmlp]

@smourier thanks for commenting here, I wasn't aware of DirectN.

[jonwis]

On the WinUI3 and Microsoft.UI.Composition front, the interop overhead should be extremely low from all runtimes using modern projections. Do C#/WinRT or C++/WinRT meet your projection needs? We aim for the projection "cost" to be extremely low and not require you to pick a specific runtime. Issue #18 asks for a "flat C" projection of metadata-based APIs for "old" callers who cannot use C++/WinRT.

@smourier - you said:

However, note that many improvements in latest Windows are not available at all even from plain C++ as "old" COM interface, only to WinRT. So I can't integrate them into that project.

Can you help me understand more? WinRT types are available to all COM-ready clients through "ABI" headers that take HSTRING and UINT32 and emit HRESULT and IAsyncAction. WinRT's based on the COM interface model, with IInspectable being the generic base type instead of IUnknown (although unsurprisingly, IInspectable derives from IUnknown.) It's still feasible, although not terribly pleasant, to use the most modern Windows Runtime APIs from assembly the same way you'd have used COM interfaces.

(I've also reached out to the DirectX team for their commentary, stay tuned)

[smourier]

@jonwis - actually, that point "So I can't integrate them into that project" is not a blocker. As you say it's possible to use WinRT from C# today.

It was more a remark about the fact that if you want to write an application that uses for example Direct2D+DirectWrite (which is a COM interface only) and Direct Composition (for which the latest version is a WinRT interface in Windows.UI.Composition), you have to mix both worlds: 1) it's quite complicated and 2) it's not really documented although there are "interop" sample about this (ICompositionDrawingSurfaceInterop).

There shouldn't be a need to use "interop" to have these world talk. It would be ok if everything was WinRT (and accessible from desktop apps) I guess.

[jonwis]

if you want to write an application that uses for example Direct2D+DirectWrite (which is a COM interface only) and Direct Composition (for which the latest version is a WinRT interface in Windows.UI.Composition), you have to mix both worlds

It would be ok if everything was WinRT (and accessible from desktop apps) I guess.

So if we had metadata-based definitions of all those APIs, and a consistent projection of that metadata to your target languages, that would help?

One of Project Reunion's explicit goals is to define as many APIs using metadata as possible, existing "flat C" or "ABI COM." Once we have that metadata tooling like the */WinRT series can project a consistent API surface for your favorite languages. This would eliminate ugly "code seams" where you have to transition from C# to ABI COM to WinRT to p/invoke.

[pjmlp]

@jonwis Basically what I am voicing is a recurring request since the introduction of WinRT and rampdown of Managed DirectX and XNA frameworks from the respective teams.

Since the introduction of WinRT there has been a barrier for certain APIs, like DirectX, whose access is only possible via C++, instead of making it visible via WinRT projections.

So since Windows 8, accessing DirectX from .NET has been a mix of:

• Just use a wrapper library like SharpDX. Usually community driven, playing catch up with official API, most of them one person show, and eventually unmaintained (like it has happened to SharpDX);

• Make use of a game engine like Unity or Stride (former Xenko). Not really something desirable when the goal is to integrate DirectX capabilities into regular LOB applications;

• Make use of Win2D. Only usable for 2D scenarios and it is unclear how much we should rely on it going forward

• Write the bindings ourselves in C++. Usually a pain for those .NET developers that aren't confortable using C++, an alternative that has gotten worse with the dismissal of C++/CX, given that C++/WinRT tooling is still found lacking versus C++/CX (no VS syntax highlighting and completion support for IDL files, manually copying generated files into VS projects, manual boilerplate for XAML integration)

With the evolution of UWP, the Windows.Graphics.DirectX.Direct3D11 finally appeared, but it is only meant for interoperability with Visual Layer and little else.

Ideally DirectX would be fully exposed under Windows.Graphics.DirectX.Direct3D11 and Windows.Graphics.DirectX.Direct3D12 and so forth, so that any .NET developer could be able to directly access DirectX capabilities without having to learn C++ or rely on game engines for LOB scenarios.

Same applies to other Windows APIs that aren't currently exposed via WinRT projections.

Naturally I am not placing this issue for myself alone, I already have my solution in place, just as something that I think the Windows developer community at large would benefit from, regardless of which language projection they make use of.

Now if this is something that Project Reunion projections will enable out of the box, then it would already be a quite good solution.

[jonwis]

Thanks for all the discussion on this issue! I talked to Shawn on the DirectX team about this and they say:

This is an interesting topic but we’re mostly focusing on native clients right now. DirectX’s API has a number of constructs that don’t easily port to non-native runtimes like JS or C#. We suggest you use a layered approach where your direct calls into the DirectX API surface are from C++ and you provide an app-specific interface from that component into your application.

We’re happy to help you use Project Reunion tooling – WinMDs, C++/WinRT implementation of projections, metadata-based projections to other languages – to produce that glue layer, and if it evolves over time to meet more apps needs we’d welcome it as a Project Reunion Family member. Check out https://github.com/microsoft/RegFree_WinRT or other http://github.com/microsoft/xlang samples for how to create IDL-based object definitions and a CLR-ready projection.

Your best bet out - right now - for GPU-accelerated 2D rendering (or inspiration for your wrapper) is https://github.com/microsoft/Win2D

@smourier - could your tool produce IDLs, then an implementation with C++/WinRT, and then use C#/WinRT to access it from the CLR?

[jonwis]

Shawn also suggests checking out their Discord - https://discord.com/invite/directx for further discussion.

[smourier]

I think the question was not "how can we do that?". But more "we know how to do it today, but it's far from being straightforward, please include DirectX in your thinking".

Speaking for myself, I know how to do it today, not specifically looking for hints nor tech support, more for API unification which I though Reunion was about.

Now you're making clear DirectX (and friends) are not part of project Reunion, Duh....

Win2D is only for UWP, it's not really a product as such (it has its own doc, it's not really updated), plus it has some terrible dependencies over VCRT, so it's nothing but a general answer. In fact I though Reunion was about that: getting rid of "things" such as Win2D.

[pjmlp]

Basically it means that despite WinRT and UWP, the DirectX team has no plans to support anything beyond C++ and plain old COM, and it is up for us to do the work for free as Microsoft cannot afford it.

I wonder what is the goal of Project Reunion after all, or even how Rust/WinRT will also have their own community provided bindings for DirectX.

Well, I guess it is settled then, thanks for clarifying, at least it wasn't closed right away.

[Gavin-Williams]

If DirectX is left behind as a legacy system, it makes C# and Direct X both weaker. If I can't use DirectX from C# I don't really need or want either of them. Currently, there is no mature and supported project that allows C# and Dx to be used together (SharpDX is retired) and nothing else provides the same experience.

Are we being told not to write performant graphics software for Windows anymore please? Unity seems to be the only real option, and that's not really suitable for certain kinds of games/apps. I just can't read it any other way. You make C# the # 1 language on Windows, with the best API's, and the best tooling, but you leave out one of the core components - graphics. I don't get it.

[kennykerr]

There is now metadata for Win32 APIs including DirectX:

Repo: https://github.com/microsoft/win32metadata

All you need is a language projection that will generate bindings for your preferred language. DirectX already works quite well in Rust.

Repo: https://github.com/microsoft/windows-rs DirectX examples: https://github.com/microsoft/windows-samples-rs

There is also a C# language projection based on the same Win32 metadata:

Repo: https://github.com/microsoft/cswin32

I'm not sure whether they have DirectX working yet but you can ask.

[smourier]

C# projections have their share of issues especially with low-level technology such as DirectX and friends https://github.com/microsoft/CsWin32/issues/167 https://github.com/microsoft/CsWin32/issues/26 eventually this will converge someday.

I'm also wondering about the official status of projects like these, like are they here to stay is this part of the "reunion"? or just "skunk" projects (remembering the defunct Managed DirectX) ?

[jtorjo]

@kennykerr @smourier I've created a ticket https://github.com/microsoft/CsWin32/issues/327 -- lets see how it goes

[Gavin-Williams]

@kennykerr thanks for the info. But all of that is over my head. I don't know much about metadata and I know nothing about using it for anything - it's something the compiler uses isn't it?

"All you need is a language projection that will generate bindings for your preferred language." - I've taken a look at TerraFX https://github.com/terrafx - And yes, I can call DirectX from C# but it's all unsafe and janky DirectX style, which doesn't sit well with C# in my mind. I do understand that it might be suitable for C++ dev's who want to make C# wrappers and engines though. It's just not for the everyday joe needing performant 2d/3d graphics. And another concern with anything unofficial is no guarantee that it's going to be supported in 12 months time.

I really want a C# style wrapper. Modern, non-verbose, organized hierarchically with short names, consistent argument types (eg C# style enums where appropriate). There's just some really weird stuff in DirectX. C++ & DirectX together have a lot of qualities that are rather unpleasant for C# devs.

TerraFX and I suppose any straight projection is going to bring all that stuff with it. We just really want an easy and natural way to use DirectX, or even better, a C# style graphics library that competes with DirectX (80% of it's framerate would do).

I think you understand how I feel because you did that C++ DirectX wrapper a few years ago - dx.h. That must have come from a sense that things could be better. There are difficulties with DirectX that can't be communicated to most C++/DirectX dev's, they are so accustomed to using DirectX the way it is, they seem to not understand any criticisms. Or dismiss them with 'it's too difficult to change things because of nano-com' or I get told to 'use something else if you don't like it'.

SharpDX was brilliant, if It was taken up by Microsoft and modernized with the new C# stuff like function pointers and the rest of the modern C# goodies (.Net 5), I would just use that. Most games and graphics could be written in C#, only the very top tier of games would require C++/DirectX.

I'm currently having a go at writing a WinRT component wrapping Dx12. It's difficult, I'm not a C++ programmer. And because I had to abandon SharpDX, I have mostly floundered for the past 18 months, I'm learning C++ because it seems I don't have a choice.

[kennykerr]

I think you understand how I feel because you did that C++ DirectX wrapper a few years ago - dx.h. That must have come from a sense that things could be better.

Yep! 😉

SharpDX was brilliant

Yep, @xoofx did a great job.

Yes, I see you're after a modern language wrapper for DirectX - not merely the raw bindings. The new metadata provides the raw bindings that tools like SharpDX could use to produce more idiomatic bindings, but it's not directly useful for app or game developers.

I'm learning C++ because it seems I don't have a choice.

While I would never discourage anyone from learning C++, it's a fun language, might I suggest Rust as a modern alternative to C++. It solves so many of the tooling, language, and packaging issues that are endemic to the C++ ecosystem, while retaining the performance characteristics of the language.

https://www.rust-lang.org/

[Gavin-Williams]

I've been wondering about Rust. There seems to be a bit of effort to provide it as a supported language on Windows. So I will definitely check it out, Thank you.

[orcmid]

We suggest you use a layered approach where your direct calls into the DirectX API surface are from C++ and you provide an app-specific interface from that component into your application.

I realize this is a pain point for developers who want close-to-hardware support for graphics-intensive applications in their chosen languages.

The desire for re-abstraction of DirectX to a higher-level API/language scheme may be a great example of "no good deed goes unpunished." The whole point was to get to the nitty-gritty in a way that made more of the GPU available to native code that is amenable to the structures and methods involved. Some degree of device dependency might also arise though it seems to be handled by specifying specific minimum DirectX version dependency.

Elevating DirectX becomes an over-constrained problem. The suggestion is to build ones own elevations that are perfect for the intended application and do not attempt to deliver it all. That is pretty typical of what is done in these cases. And yes, there might need to be versioned support as DirectX evolves. It is also not easy. So there are exepertise considerations and constraints as well.

I take that to be a demonstration of how even more difficult a general solution for higher-level, still high-performing DirectMumble is.

PS: Is the SharpDX code not available for treatment as a reference and/or reverse-engineering?