Add a WinUI3 backend

[jhi2]

(Ticket content has been edited significantly from the original to provide better guidance on the feature request)

What is the problem or limitation you are having?

Toga's look and feel on Windows currently uses Winforms, which doesn't reflect the current look and feel of Windows apps.

Describe the solution you'd like

We should add a WinUI3 backend.

Describe alternatives you've considered

Continue to use Winforms.

Additional context

The prerequisite for this backend is the development of Python bindings for WinUI3.

[freakboy3742]

Thanks for the suggestion - we'd love to add a WinUI3 backend; the problem is that there are no Python bindings for WinUI3. Once those bindings exist and are stable, a WinUI3 backend for Toga can be developed.

[jhi2]

Maybe I could help make one

[JPHutchins]

Hi @jhi2, I am a C programmer and would be happy to help. In this case, I think that the ctypes + DLL approach taken by win32more makes sense, at least to get an MVP. Generally, I am having a hard time understanding that repo, it is very complex. It sorta looks like a code generation tool PLUS some generated code, which makes usage confusing. Plus, organizing all these namespaces by way of directories is perhaps nice for repo organization but it may be not so great for Python import initialization. I have concerns about the "DLL function decorator". Perhaps win32more can be used to generate only the bindings that a Toga backend would need?

Happy to help with testing or other tasks.

TBH, anything other than a Microsoft supported solution will be a bit of a hack. MS probably doesn't have the money or talent for this 🙄.

[JPHutchins]

To see how simple the ctypes APIs can be, try this example:

import ctypes as c

user32 = c.WinDLL("user32.dll")

user32.MessageBoxW(0, "Hello World", "Hello", 1)

Where the documentation is from here: https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-messageboxw

The tedious bit is having nice wrappers with typing.

I am looking into the winrt package collection: https://pypi.org/search/?q=winrt&page=1

So far I have not been able to reproduce a simpler version of what win32more is doing. I am able to import an XAML Application class but cannot create the application.

[JPHutchins]

The win32more interface to Application seems different than WinRT: https://github.com/ynkdir/py-win32more/blob/87f331b014fd1cda3ede67c4cf10715acafe575a/win32more/Microsoft/UI/Xaml/__init__.py#L67-L130

Compare to the interface generated by WinRT:

@typing.final
class Application_Static(type):
    @typing.overload
    def load_component(cls, component: typing.Optional[winrt.system.Object], resource_locator: typing.Optional[windows_foundation.Uri], /) -> None: ...
    @typing.overload
    def load_component(cls, component: typing.Optional[winrt.system.Object], resource_locator: typing.Optional[windows_foundation.Uri], component_resource_location: microsoft_ui_xaml_controls_primitives.ComponentResourceLocation, /) -> None: ...
    def start(cls, callback: typing.Optional[ApplicationInitializationCallback], /) -> None: ...
    @_property
    def current(cls) -> typing.Optional[Application]: ...

@typing.final
class Application(winrt.system.Object, metaclass=Application_Static):
    @staticmethod
    def _from(obj: winrt.system.Object, /) -> Application: ...
    def __new__(cls: typing.Type[Application]) -> Application: ...
    def exit(self) -> None: ...
    def add_unhandled_exception(self, handler: typing.Optional[UnhandledExceptionEventHandler], /) -> windows_foundation.EventRegistrationToken: ...
    def remove_unhandled_exception(self, token: windows_foundation.EventRegistrationToken, /) -> None: ...
    def add_resource_manager_requested(self, handler: windows_foundation.TypedEventHandler[winrt.system.Object, ResourceManagerRequestedEventArgs], /) -> windows_foundation.EventRegistrationToken: ...
    def remove_resource_manager_requested(self, token: windows_foundation.EventRegistrationToken, /) -> None: ...
    @_property
    def resources(self) -> typing.Optional[ResourceDictionary]: ...
    @resources.setter
    def resources(self, value: typing.Optional[ResourceDictionary]) -> None: ...
    @_property
    def requested_theme(self) -> ApplicationTheme: ...
    @requested_theme.setter
    def requested_theme(self, value: ApplicationTheme) -> None: ...
    @_property
    def high_contrast_adjustment(self) -> ApplicationHighContrastAdjustment: ...
    @high_contrast_adjustment.setter
    def high_contrast_adjustment(self, value: ApplicationHighContrastAdjustment) -> None: ...
    @_property
    def focus_visual_kind(self) -> FocusVisualKind: ...
    @focus_visual_kind.setter
    def focus_visual_kind(self, value: FocusVisualKind) -> None: ...
    @_property
    def debug_settings(self) -> typing.Optional[DebugSettings]: ...
    @_property
    def dispatcher_shutdown_mode(self) -> DispatcherShutdownMode: ...
    @dispatcher_shutdown_mode.setter
    def dispatcher_shutdown_mode(self, value: DispatcherShutdownMode) -> None: ...

Here's the C# doc: https://learn.microsoft.com/en-us/windows/windows-app-sdk/api/winrt/microsoft.ui.xaml.application?view=windows-app-sdk-1.5

[freakboy3742]

If you're calling APIs in user32, you're not calling WinUI3 API; you're calling the 1995 era win32 API. It's also available using the win32api package, which is mostly a set of convenience wrappers around the win32 API.

It may be possible to use ctypes to call WinUi3 libraries, but the API calls and libraries will be different.

[JPHutchins]

It may be possible to use ctypes to call WinUi3 libraries, but the API calls and libraries will be different.

Yes! The intention is to show how easy it is to make calls to any DLL from python.

Presently I'm trying to track down how win32more is hooking into the new APIs. This decorator is on Application::Start:

https://github.com/ynkdir/py-win32more/blob/87f331b014fd1cda3ede67c4cf10715acafe575a/win32more/_winrt.py#L710-L724

It's getting the "Start" function from this:

https://github.com/ynkdir/py-win32more/blob/87f331b014fd1cda3ede67c4cf10715acafe575a/win32more/_winrt.py#L786-L793

Seems like the magic is in the RoGetActivationFactory, which I've not heard of before: https://learn.microsoft.com/en-us/windows/win32/api/roapi/nf-roapi-rogetactivationfactory

[JPHutchins]

xaml = ctypes.WinDLL("windows.ui.xaml.dll")

Is actually loading something... not sure what - can't find any function pointers.

[freakboy3742]

If you're digging into this, I'd pay particular attention to this comment on the WinUI3 ticket I've referenced earlier in this discussion. @zooba is a Microsoft employee and member of the CPython core team; his offer around co-developing bindings should not be ignored or taken lightly.

[zooba]

Yeah, this is a really deep hole to dive into :) I was lucky to spend my intern project at MSFT (in 2011) working on this stuff while the APIs were being developed, and I don't think any of the low-level intro material we had ever made it public.

Basically, all the WinRT APIs are approx. COM, which means you don't load them as normal C APIs. The RoGetActivationFactory[^1] returns an interface that has function pointers to functions that will create instances of particular objects. Those then have interfaces to access and use those objects (as well as reference counting). It's a very powerful model, particularly for forwards and backwards binary compatibility - something of a holy grail in OS design - but it takes a bit to get it all into your head.

WinUI3 is based around these same APIs, but are registered somewhat differently. I haven't dug too far into all the details, but it's more convenient (and more wasteful of memory and disk space) than the original Windows 8 UWP APIs.

But having now implemented 4 different approaches of accessing it from Python, the one I think is best is to just generate C++ code to do all the GUI side and have it call back into Python. I've implemented something along these lines here[^2]. It was pretty easy to translate some existing C# samples into Python, but I haven't really tried it at scale (though it already scales so much better than previous efforts that I'm sure it'd be fine). There might be something of value in there for you.

[^1]: IIRC, "RO" stood for Runtime Object. The equivalent COM APIs start with "CO".

[^2]: Based on my pymsbuild build backend, since building is an unavoidable part of code generation.

[jhi2]

Hi @jhi2, I am a C programmer and would be happy to help. In this case, I think that the ctypes + DLL approach taken by win32more makes sense, at least to get an MVP. Generally, I am having a hard time understanding that repo, it is very complex. It sorta looks like a code generation tool PLUS some generated code, which makes usage confusing. Plus, organizing all these namespaces by way of directories is perhaps nice for repo organization but it may be not so great for Python import initialization. I have concerns about the "DLL function decorator". Perhaps win32more can be used to generate only the bindings that a Toga backend would need?

Happy to help with testing or other tasks.

TBH, anything other than a Microsoft supported solution will be a bit of a hack. MS probably doesn't have the money or talent for this 🙄.

I like the "MS probably doesn't have the money or talent for this 🙄" That's why they invented Rectify11 haha. I have a project on this and I may need to make my own binding. Your help would be appreciated! see:https://github.com/jhi2/PyUi3 NOTE:I only know core C++.

[jhi2]

windows.ui.xaml.dll is for WinForms or WPF Microsoft.Ui.Xamlis for WinUI

[jhi2]

Yeah, this is a really deep hole to dive into :) I was lucky to spend my intern project at MSFT (in 2011) working on this stuff while the APIs were being developed, and I don't think any of the low-level intro material we had ever made it public.

Basically, all the WinRT APIs are approx. COM, which means you don't load them as normal C APIs. The RoGetActivationFactory1 returns an interface that has function pointers to functions that will create instances of particular objects. Those then have interfaces to access and use those objects (as well as reference counting). It's a very powerful model, particularly for forwards and backwards binary compatibility - something of a holy grail in OS design - but it takes a bit to get it all into your head.

WinUI3 is based around these same APIs, but are registered somewhat differently. I haven't dug too far into all the details, but it's more convenient (and more wasteful of memory and disk space) than the original Windows 8 UWP APIs.

But having now implemented 4 different approaches of accessing it from Python, the one I think is best is to just generate C++ code to do all the GUI side and have it call back into Python. I've implemented something along these lines here2. It was pretty easy to translate some existing C# samples into Python, but I haven't really tried it at scale (though it already scales so much better than previous efforts that I'm sure it'd be fine). There might be something of value in there for you.

Footnotes

1. IIRC, "RO" stood for Runtime Object. The equivalent COM APIs start with "CO". ↩
2. Based on my pymsbuild build backend, since building is an unavoidable part of code generation. ↩

I will use cTypes

[JPHutchins]

I like the "MS probably doesn't have the money or talent for this 🙄" That's why they invented Rectify11 haha.

Literally, cannot wait until AI is "over". But then there will be the "next thing" that takes resources away from luxuries like being able to produce and distribute signed applications.

windows.ui.xaml.dll is for WinForms or WPF Microsoft.Ui.Xamlis for WinUI

Will take a look!

Also, consider testing this as the backend instead of win32more: https://github.com/zooba/pymsbuild-winui

[JPHutchins]

Very maybe we can get ctypes - I'll have to fiddle.

>>> com = ctypes.WinDLL("combase.dll")
>>> com.RoGetActivationFactory
<_FuncPtr object at 0x000001F9BE423790>

[JPHutchins]

No, it still seems foolish to try: https://learn.microsoft.com/en-us/windows/win32/api/roapi/nf-roapi-rogetactivationfactory#syntax

The RoGetActivationFactory1 returns an interface that has function pointers to functions that will create instances of particular objects. Those then have interfaces to access and use those objects (as well as reference counting).

[jhi2]

I would just want to call C++ code from ctypes. I found a way to call Python from C/C#/C++(Python.h)

[jhi2]

Literally, cannot wait until AI is "over". But then there will be the "next thing" that takes resources away from luxuries like being able to produce and distribute signed applications.

Really? how will u sign them? Linux?

[JPHutchins]

I would just want to call C++ code from ctypes. I found a way to call Python from C/C#/C++(Python.h)

This might be nice, but I'll recommend pymsbuild-winui or win32more for generating the bindings. I think it might be possible with ctypes but feels hacky. I mean, it's all hacky. That RoGetActivationFactory will provide a function pointer, but I'm not sure how to bind it to a python function. Maybe with this: https://docs.python.org/3/library/ctypes.html#callback-functions

[zooba]

You'll actually want function prototypes, but you might also be able to find a COM library that implements the core functionality (e.g. the IUnknown interface, which is the default function table that every interface starts with).

(And to be clear, I'm only recommending this for research purposes. There's just no way to build everything you'll need with ctypes, but you can certainly explore a bit and there's a ton of interesting stuff to learn!)

[JPHutchins]

Presently leaning toward a proof of concept that is a PyPI package with static libs based on pymsbuild-winui.

One goal is to avoid developers needing to install the VS toolchains.

[jhi2]

Also, consider testing this as the backend instead of win32more: https://github.com/zooba/pymsbuild-winui Will try that

[jhi2]

I DO NOT LIKE the setup of pymsbuild-winui

[JPHutchins]

I DO NOT LIKE the setup of pymsbuild-winui

It's not setup as a package yet, so that would be the work, AFAICT. I really hope to have some time to look into this soon!

[jhi2]

I have a remarkable new breakthrough, based on the foundation that WinUI3 is a COM object. I will come out with the code soon here.

[jhi2]

Here is the code I promised:

pip install pywin32 

import win32com.client
Microsoft.Ui.Xaml = win32com.client.Dispatch("Windows.Ui.Xaml")
#WinUI3 is now accessable

[freakboy3742]

Anyone what? What response are you expecting?

You've posted a snippet of code that suggests that the WinUI3 API may be accessible - using an approach that didn't occur to a Microsoft employee who worked on the problem while working at Microsoft.

Even if we assume that the approach you've described does work - there is a huge gap between "here's how you can access the WinUI3 libraries" and "Here's a WinUI3 backend for Toga.". It definitely didn't happen in the 30 minutes between you posting the proposed solution and responding "Anyone?".

[jhi2]

Ok

[jhi2]

Can you look it over?

[JPHutchins]

Can you look it over?

I can look it over. Think of this as an exploratory phase where we're looking for cleanest starts of a minimum viable product.

Ideally, we reproduce this in "pure" python: https://github.com/sotanakamura/winui3-without-xaml

I'll be fiddling presently, so feel free to send examples of what you have so far.

[jhi2]

Can't the Toga code generate the xaml?

[JPHutchins]

Can't the Toga code generate the xaml?

That would be one approach.

[freakboy3742]

Look over what? You've provided 3 lines of code.

I'm sceptical that the approach you've described will work - because if something that simple would work, I feel like @zooba would have suggested it. However, I have no particular expertise with COM, WinUI3, or Windows in general, so I'm not really in a position to provide any sort of critical analysis of the approach.

Adding a WinUI3 backend isn't on my roadmap for this quarter - nor is it likely to be on my roadmap in the foreseeable future. If this is something that interests you, than I'm able to provide advice on how to go about building a WinUI3 backend; and I'm willing to review code in the form of PRs against the Toga repo - but until there's working WinUI3 code that can run, there's not much more I'm in a position to offer.

And - answering your follow up question - no, Toga can't use a XAML approach. Toga needs to be able to programmatically drive the construction of widgets. Generating markup isn't an approach that is fundamentally compatible with Toga.

[jhi2]

Can you look it over?

I can look it over. Think of this as an exploratory phase where we're looking for cleanest starts of a minimum viable product.

Ideally, we reproduce this in "pure" python: https://github.com/sotanakamura/winui3-without-xaml

I'll be fiddling presently, so feel free to send examples of what you have so far.

Traceback (most recent call last): File "", line 1, in File "C:\Users\johnd\AppData\Roaming\Python\Python312\site-packages\win32com\client__init__.py", line 118, in Dispatch dispatch, userName = dynamic._GetGoodDispatchAndUserName(dispatch, userName, clsctx) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\johnd\AppData\Roaming\Python\Python312\site-packages\win32com\client\dynamic.py", line 104, in _GetGoodDispatchAndUserName return (_GetGoodDispatch(IDispatch, clsctx), userName) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\johnd\AppData\Roaming\Python\Python312\site-packages\win32com\client\dynamic.py", line 86, in _GetGoodDispatch IDispatch = pythoncom.CoCreateInstance( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ pywintypes.com_error: (-2147221005, 'Invalid class string', None, None)

[jhi2]

Is there any way to fix this

[JPHutchins]

Is there any way to fix this

I was hoping that you already had 🤣! I am seeing the same thing, and I suspect that it's (one of) the reason that we're being steered towards alternatives by zooba.

[JPHutchins]

@jhi2 Found this little tool: python -m win32com.client.combrowse

[JPHutchins]

I have built the example applications from https://github.com/zooba/pymsbuild-winui

Unfortunately I didn't take notes, but after some fiddling, I'd suggest the following workflow:

• add .venv to the .gitignore
• python -m venv .venv
• . .venv/scripts/activate
• pip install .
• pip install -r tests/requirements.txt
• pytest
• I did get a working executable at pymsbuild-winui\tests\testdata\app1\build\bin\TestApp\app.exe

Other thoughts

• (PhotoViewer exe didn't work - it didn't move the python dlls and stuff into the app folder for some reason)
• build is putting a lot of files in tree instead of inside the build or temp folder which is annoying
• the TestApp folder is 228 MB - might be good though - the microsoft DLLs are getting copied over

Next steps

• Create a package using pymsbuild that provides a hello world to some single WinUI3 API call

[jhi2]

@jhi2 Found this little tool: python -m win32com.client.combrowse

Nice!

[jhi2]

have you seen https://pypi.org/project/winrt/

[freakboy3742]

@jhi2 The Python WinRT projection is the subject of literally the first comment on the WinUI3 thread. AIUI, it can't be used for exposing WinUI3 APIs because you can't build an automated binding to C++.

[jhi2]

have you seen:

1. https://pywinrt.readthedocs.io/en/latest/api/microsoft.html
2. https://learn.microsoft.com/en-us/windows/windows-app-sdk/api/winrt/

[freakboy3742]

Can't say I've seen those. Last time I looked, winrt didn't have any GUI bindings at all; and from my discussions with people at MS, it wasn't plausible to add them, either. If that isn't the case any more, then it might be possible to build a WinUI3 backend using winrt as the binding.

[JPHutchins]

have you seen:

1. https://pywinrt.readthedocs.io/en/latest/api/microsoft.html

2. https://learn.microsoft.com/en-us/windows/windows-app-sdk/api/winrt/

winrt was my original starting point and I was unable to replicate the win32more example.

Specifically, this call was my stopping point: https://learn.microsoft.com/en-us/windows/windows-app-sdk/api/winrt/microsoft.ui.xaml.application.start?view=windows-app-sdk-1.5#microsoft-ui-xaml-application-start(microsoft-ui-xaml-applicationinitializationcallback)

[jhi2]

Hey, for my app, can you make a win32more-like binding for my app for pymsbuild-winui. submit as pr to jhi2/PyUI3

[freakboy3742]

@jhi2 I'm not looking for more projects to contribute to. I'd certainly like a WinUI3 binding to exist, but it's not a priority for me at present, partially because Toga has a binding that works on Windows, and partially because I don't use Windows as a platform myself. If you want a WinUI3 binding, you'll need to either develop it yourself, pay for someone else to develop it for you, or wait for someone else to develop it for you.

[JPHutchins]

Hey, for my app, can you make a win32more-like binding for my app for pymsbuild-winui. submit as pr to jhi2/PyUI3

Go ahead and tag me on a discussion or issue on that repo to continue this discussion. I should not have criticized win32more so much... it does seem to be providing a programmatic interface to WinUI3 which is more than other attempts have yielded. Perhaps it's complexity is necessary and something that can be improved over time. Anyway, keep going with that and see what you can do! But let's work outside of Toga until there's something to offer Toga.

[jhi2]

Yeah, an error. I'll tag u.

[KubaO]

Can you look it over?

I can look it over. Think of this as an exploratory phase where we're looking for cleanest starts of a minimum viable product.

Ideally, we reproduce this in "pure" python: https://github.com/sotanakamura/winui3-without-xaml

I'll be fiddling presently, so feel free to send examples of what you have so far.

pywin32more makes it a solved problem. It's an impressive Python binding/projection for most of WinApi, including the core bits from the 90s, as well as WinUI3, and most everything else that's available via COM.

What's even more impressive is that it's a Python-only package. It does everything via ctypes. On top of those it adds runtime type checking, and has type annotations for the entire API. Of course it's machine generated, and the generator is quite clever. The projection is lazy-loaded, so that you only pay for the identifiers that you use and nothing more.

Note that XamlApplication is an Application that supports XAML, but it doesn't need to use XAML at all. Everything can be pure Python :)

from win32more.xaml import XamlApplication
from win32more.Windows.Foundation import Uri
from win32more.Microsoft.UI.Xaml import (
    HorizontalAlignment,
    ThicknessHelper,
    VerticalAlignment,
    Window,
    )
from win32more.Microsoft.UI.Xaml.Controls import (
    Button,
    HyperlinkButton,
    StackPanel,
    TextBlock,
    )
from win32more.Microsoft.UI.Xaml.Media import (
    DesktopAcrylicBackdrop,
    )

class App(XamlApplication):
    def OnLaunched(self, args):
        window = Window()
        self.window = window
        window.SystemBackdrop = DesktopAcrylicBackdrop()

        stackPanel = StackPanel()
        stackPanel.HorizontalAlignment = HorizontalAlignment.Center
        stackPanel.VerticalAlignment = VerticalAlignment.Center

        title = TextBlock()
        title.Style = App.Current.Resources.Lookup("TitleTextBlockStyle")
        title.Text = "WinUI 3 in Python Without XAML!"
        title.HorizontalAlignment = HorizontalAlignment.Center

        project = HyperlinkButton()
        project.Content = "Github Project Repository"
        project.NavigateUri = Uri("https://github.com/sotanakamura/winui3-without-xaml")
        project.HorizontalAlignment = HorizontalAlignment.Center

        button = Button()
        button.Content = "Click Me"
        button.Click += lambda sender, args: button.put_Content("Thank You!")
        button.HorizontalAlignment = HorizontalAlignment.Center
        button.Margin = ThicknessHelper.FromUniformLength(20);

        window.Content = stackPanel
        stackPanel.Children.Append(title)
        stackPanel.Children.Append(project)
        stackPanel.Children.Append(button)

        window.Activate()

    def ButtonClick(self, sender, args):
        sender.as_(Button).Content = "Thank You!"

if __name__=="__main__":
    XamlApplication.Start(App)

[KubaO]

I would just want to call C++ code from ctypes. I found a way to call Python from C/C#/C++(Python.h)

This might be nice, but I'll recommend pymsbuild-winui or win32more for generating the bindings. I think it might be possible with ctypes but feels hacky. I mean, it's all hacky. That RoGetActivationFactory will provide a function pointer, but I'm not sure how to bind it to a python function. Maybe with this: https://docs.python.org/3/library/ctypes.html#callback-functions

win32more uses ctypes only. It's pure Python, machine-generated using a generator written in Python from JSON API description available elsewhere. Pretty much all of Windows API that is available from C is covered, and WinUI is definitely available via C, since COM works in C just fine, albeit it's a pain to use without generated code.