Unity3D support

[VincentMarnier]

Is your feature request related to a problem? Please describe. I would like to use Webview2 in my Unity3D app on windows systems. I imported the Core dll in my project, and I also use the WebView2Loader.dll. Anytime I call CoreWebView2Environment.CreateAsync();, the Unity Editor (or built project) crash. I have read in the issue #401 that back in 2020 you were aiming to support Unity3D in 2021. Am I missing something? If not, do you still plan such a support?

Describe the solution you'd like and alternatives you've considered Idealy the solution I would like is a WebView2 that works in Unity3D games. The alternative I am considering right now is to try with the Win32 lib, make some wrapper and try to call native functions from Unity myself; but I believe it will fail, I am pretty sure it is already what your Core library does.

AB#40880023

[nishitha-burman]

Hello,

We are tracking (and have started) the work in https://github.com/MicrosoftEdge/WebView2Feedback/issues/20. Does this address your requirements? Thanks!

[VincentMarnier]

Hello @nishitha-burman, First of all, thanks a lot for your answer and thanks for pointing #20. In many ways it should have addressed my requirements. I can see some workaround mentionned there that consist of retrieving the render of WebView2, this could then be used in Unity environment as a texture. I guess that what would be left to do is to forward inputs to the WebView2 component and it would definetly work as expected.

Yet I think my issues are of different kind. I have two of them.

As mentionned, the first one is that the Core lib (.NET Framework) seems to not work with Unity, it crashes the Unity Editor and the Unity builds when we try to create an environment. It seems that there is a compatibility issue, which seems weird, I might investigate more on that later.

The second issue is that the solutions mentioned in #20 seems not applicable to Unity at the end of the day. I created my own minimal C++ lib that uses WebView2, I made some C# Interop Wrapper and stuff and I then try to render something inside of Unity.

Click to see my C# code (it is a bit dirty, sorry)

```using System; using System.IO; using System.Runtime.InteropServices; using System.Runtime.InteropServices.ComTypes; using UnityEngine; using UnityEngine.UI; public class TestWebView2 : MonoBehaviour { private enum COREWEBVIEW2_CAPTURE_PREVIEW_IMAGE_FORMAT { PNG = 0, JPEG = 1 } private class NativeStream : IStream, IDisposable where T : Stream, new() { public T ManagedStream; public NativeStream() : this(new T()) { } public NativeStream(T stream) { ManagedStream = stream; } public void Dispose() { ManagedStream.Dispose(); } public void Clone(out IStream ppstm) { T cloneStream = new T(); try { ManagedStream.CopyTo(cloneStream); ppstm = new NativeStream(cloneStream); } catch { cloneStream.Dispose(); throw; } } public void Commit(int grfCommitFlags) { throw new NotSupportedException(); } public void CopyTo(IStream pstm, long cb, IntPtr pcbRead, IntPtr pcbWritten) { byte[] buffer = new byte[cb]; int bytesReaden = ManagedStream.Read(buffer, 0, Convert.ToInt32(cb)); if (pcbRead != IntPtr.Zero) { Marshal.WriteInt32(pcbRead, bytesReaden); } pstm.Write(buffer, bytesReaden, pcbWritten); } public void LockRegion(long libOffset, long cb, int dwLockType) { throw new NotSupportedException(); } public void Read(byte[] pv, int cb, IntPtr pcbRead) { Int32 bytesRead = ManagedStream.Read(pv, 0, (int)cb); if (pcbRead != IntPtr.Zero) { Marshal.WriteInt32(pcbRead, bytesRead); } } public void Revert() { throw new NotSupportedException(); } public void Seek(long dlibMove, int dwOrigin, IntPtr plibNewPosition) { SeekOrigin seekOrigin; switch (dwOrigin) { case 0 /* STREAM_SEEK_SET */: seekOrigin = SeekOrigin.Begin; break; case 1 /* STREAM_SEEK_CUR */: seekOrigin = SeekOrigin.Current; break; case 2 /* STREAM_SEEK_END */: seekOrigin = SeekOrigin.End; break; default: throw new ArgumentOutOfRangeException(nameof(dwOrigin)); } long position = ManagedStream.Seek(dlibMove, seekOrigin); if (plibNewPosition != IntPtr.Zero) { Marshal.WriteInt64(plibNewPosition, position); } } public void SetSize(long libNewSize) { ManagedStream.SetLength(libNewSize); } public void Stat(out System.Runtime.InteropServices.ComTypes.STATSTG pstatstg, int grfStatFlag) { pstatstg = new System.Runtime.InteropServices.ComTypes.STATSTG() { cbSize = ManagedStream.Length, grfMode = 0, type = 2 /* STGTY_STREAM */ }; if (ManagedStream.CanRead && ManagedStream.CanWrite) { pstatstg.grfMode |= 0x00000002 /* STGM_READWRITE */; } else if (ManagedStream.CanRead) { pstatstg.grfMode |= 0x00000000 /* STGM_READ */; } else if (ManagedStream.CanWrite) { pstatstg.grfMode |= 0x00000001 /* STGM_WRITE */; } else { throw new IOException(); } } public void UnlockRegion(long libOffset, long cb, int dwLockType) { throw new NotSupportedException(); } public void Write(byte[] pv, int cb, IntPtr pcbWritten) { ManagedStream.Write(pv, 0, cb); if (pcbWritten != IntPtr.Zero) { Marshal.WriteInt32(pcbWritten, cb); } } } private delegate void CreateUnityWebView2EnvironmentCallback(IntPtr env); private delegate void CreateUnityWebView2ControllerCallback(IntPtr controller); private delegate void UnityWebView2CapturePreviewCallback(bool result); private delegate void UnityWebView2NavigationStartingEvent(IntPtr webview, IntPtr args); private delegate void UnityWebView2NavigationCompletedEvent(IntPtr webview, IntPtr args); [DllImport("WebView2ForUnity.dll")] private static extern bool CreateUnityWebView2Environment(CreateUnityWebView2EnvironmentCallback callback); [DllImport("WebView2ForUnity.dll", CharSet = CharSet.Unicode)] private static extern bool UnityWebView2Environment_CreateController(IntPtr env, string applicationName, CreateUnityWebView2ControllerCallback callback); [DllImport("WebView2ForUnity.dll")] private static extern IntPtr UnityWebView2Controller_GetUnityWebView2(IntPtr controller); ///

/// Set bounds to the webviews managed by this controller ///

/// Pixels starting from left /// Pixels starting from top [DllImport("WebView2ForUnity.dll")] private static extern bool UnityWebView2Controller_SetBounds(IntPtr controller, long x, long y, long width, long height); [DllImport("WebView2ForUnity.dll")] private static extern bool UnityWebView2Controller_SetIsVisible(IntPtr controller, bool isVisible); [DllImport("WebView2ForUnity.dll")] private static extern bool UnityWebView2Controller_GetWindowHandle(IntPtr controller, ref IntPtr windowHandle); [DllImport("WebView2ForUnity.dll", CharSet = CharSet.Unicode)] private static extern bool UnityWebView2_Navigate(IntPtr webview, string url); [DllImport("WebView2ForUnity.dll")] private static extern bool UnityWebView2_CapturePreview(IntPtr webview, COREWEBVIEW2_CAPTURE_PREVIEW_IMAGE_FORMAT imageFormat, IStream imageStream, UnityWebView2CapturePreviewCallback callback); [DllImport("WebView2ForUnity.dll")] private static extern bool UnityWebView2_AddNavigationStarting(IntPtr webview, UnityWebView2NavigationStartingEvent callback); [DllImport("WebView2ForUnity.dll")] private static extern bool UnityWebView2_AddNavigationCompleted(IntPtr webview, UnityWebView2NavigationCompletedEvent callback); [SerializeField] private RawImage _image; private IntPtr _webview = IntPtr.Zero; private bool _imageAsked = false; private Texture2D _texture = null; void Start() { // Initialize the webview and navigate to bing CreateUnityWebView2Environment(env => { Debug.Log(env == IntPtr.Zero ? "Env init failed" : "Env init succeeded"); if (env != IntPtr.Zero) { UnityWebView2Environment_CreateController(env, Application.productName, controller => { Debug.Log(controller == IntPtr.Zero ? "Controller init failed" : "Controller init succeeded"); if (controller != IntPtr.Zero) { bool result = UnityWebView2Controller_SetBounds(controller, 0, 0, Screen.width, Screen.height); Debug.Log(result ? "SetBounds succeeded" : "SetBounds failed"); result = UnityWebView2Controller_SetIsVisible(controller, true); Debug.Log(result ? "SetIsVisible succeeded" : "SetIsVisible failed"); IntPtr windowHandle = IntPtr.Zero; result = UnityWebView2Controller_GetWindowHandle(controller, ref windowHandle); Debug.Log(result ? ("GetWindowHandle succeeded: " + windowHandle) : "GetWindowHandle failed"); IntPtr webview = UnityWebView2Controller_GetUnityWebView2(controller); result = UnityWebView2_AddNavigationStarting(webview, OnNavigationStarted); Debug.Log(result ? ("AddNavigationStarting succeeded") : "AddNavigationStarting failed"); result = UnityWebView2_AddNavigationCompleted(webview, OnNavigationCompleted); Debug.Log(result ? ("AddNavigationCompleted succeeded") : "AddNavigationCompleted failed"); result = UnityWebView2_Navigate(webview, "https://bing.com"); Debug.Log(result ? "Navigate succeeded" : "Navigate failed"); _webview = webview; } }); } }); } private void Update() { // Try to apply the webview render to a RawImage's texture if (_webview != IntPtr.Zero) { if (!_imageAsked) { Debug.Log("Will capture preview"); NativeStream stream = new NativeStream(); _imageAsked = UnityWebView2_CapturePreview(_webview, COREWEBVIEW2_CAPTURE_PREVIEW_IMAGE_FORMAT.PNG, stream, succeeded => { try { Debug.Log($"Capture preview ended (succeeded: {succeeded})"); if (succeeded) { if (_texture == null) { _texture = new Texture2D(1, 1); _image.texture = _texture; } _texture.LoadImage(stream.ManagedStream.ToArray(), false); _texture.Apply(); } } finally { stream.Dispose(); _imageAsked = false; } }); if (!_imageAsked) { Debug.Log("Capture preview failed"); stream.Dispose(); } } } } private void OnNavigationStarted(IntPtr webview, IntPtr args) { Debug.Log("Navigation started"); } private void OnNavigationCompleted(IntPtr webview, IntPtr args) { Debug.Log("Navigation completed"); } } ```

Click to see the produced logs

``` Env init succeeded Controller init succeeded SetBounds succeeded SetIsVisible succeeded GetWindowHandle succeeded: 8196352 AddNavigationStarting succeeded AddNavigationCompleted succeeded Navigate succeeded Will capture preview Capture preview failed Will capture preview Capture preview failed Will capture preview Capture preview failed Will capture preview Capture preview failed Will capture preview Capture preview failed [...] ```

It seems that:

1. AddNavigationStarting & AddNavigationCompleted events are never called
2. It aint possible to capture a preview

My best guess is that something is going wrong when trying to Navigate. I might be the one at fault though, I started trying to use WebView2 with Unity not too long ago so I aint that comfortable with WebView2 yet.

Again, thanks for your time and your kind answer. I could share the C++ lib too if needed.

[nishitha-burman]

Hi @VincentMarnier, thanks for the feedback. We've added this as a scenario on our backlog!

[MrVogorip]

Hello! I also recently started trying to use WebView2 in Unity and also faced a similar problem :0 But later I came up with the idea of using an intermediate application on WinForms to send a browser image to Unity via TCP, in some form its worked, with some nuances and bugs :)) but this frankenstein solution, I will share a simplified example, maybe someone will need it https://github.com/MrVogorip/WebView2Unity I hope in the future it will be possible to use WebView2 in Unity directly.

[VincentMarnier]

At the end of the day, we successfuly used WebView2 in Unity with a custom C++ lib. I was just using pointers instead of wil::com_ptr. Everything works fine, no need for a WinForm app. We thought about that too, but it felt unreliable for production.

This leave you only with one "issue" @nishitha-burman :

the Core lib (.NET Framework) seems to not work with Unity, it crashes the Unity Editor and the Unity builds when we try to create an environment

Again, thanks for your time.

[nishitha-burman]

Glad you were able to address most of your issues! Assigning to @novac42 to help with the remaining issues with .NET Framework not working with Unity.

[ptc-rgrasset]

@VincentMarnier Would you be able to share your code/library ? I'm really interested about it.

There is this interesting project for a general unity webview cross-platform support https://github.com/gree/unity-webview, and the missing part is Windows one.

[VincentMarnier]

Hello, Thanks for mentioning unity-webview, I already had some contact with them ,their work is incredible.

I am sorry but I won't share the code/library as I legally can't since it became proprietary code for the client I was working for and I physically can't either since I don't have the source code anymore.

I can still tell you what we came out with! We created our own C++ lib that was based on Webview2's Win32 C++ lib ( https://learn.microsoft.com/fr-fr/microsoft-edge/webview2/reference/win32/?view=webview2-1.0.1722.32 ) with everything we needed. Then we made a C# wrapper to call it directly from Unity.

I know it can be hard if you never did it so I would advise reading mono's manual about pinvoke ( https://www.mono-project.com/docs/advanced/pinvoke/ ) or using some tools (on one of my personal projects I started to use ChatGPT to wrap some native code for me, so far I found it quite successful).

I hope my answer will somewhat help you. Feel free to contact me again if you need some assistance. I am always glad to help when I can

[ptc-rgrasset]

@VincentMarnier thanks that's already really helpful information, especially confirming it is all working fine via a win32 api/plugin/wrapper approach.

[ptc-ccrabb]

@VincentMarnier are you using Unity targeting Windows Desktop or Windows Store App? I'm trying to pop up a WebView2 window on HoloLens in immersive mode -- HoloLens requires UWP to be used. I'm running into walls trying to create a Win32 window (with CreateWindow) in a UWP app. Is this something you've already succeeded in doing or have you sidestepped the problem by only targeting Windows Desktop?

[VincentMarnier]

Hey, it was Windows Desktop all along. I am unsure of the possibility to achieve it using the Win32 C++ lib with UWP. I also developped with Hololens but never had to perform such tricks so there is not much I could do to help you there, sorry.

Buuuut this was posted yesterday: https://blogs.windows.com/windowsexperience/2023/04/13/microsoft-brings-windows-11-to-hololens-2/

In addition, as part of the upgrade, we’re excited to share that the latest developer tools, like Microsoft Edge WebView2 control, will now be available in preview on HoloLens 2. This gives dev teams the ability to embed web technologies (including HTML, CSS and JavaScript) into their native applications.

I think you might want to take a look at it as it will be the official way to achieve what you are trying anytime soon :)

[rehberim360]

The year is 2024. It's a shame that Microsoft can't provide a cross-platform Edge for Unity 3D developers that gets updates simultaneously with its universal versions. However, I think it is entirely due to Microsoft's lack of employees in this department that it misses the development of Edge usage and even the endless possibilities of Unity. It's really sad...

[SoylentGraham]

I can still tell you what we came out with! We created our own C++ lib that was based on Webview2's Win32 C++ lib ( https://learn.microsoft.com/fr-fr/microsoft-edge/webview2/reference/win32/?view=webview2-1.0.1722.32 ) with everything we needed. Then we made a C# wrapper to call it directly from Unity.

Do you remember if you managed a HWND within unity, or if you used CapturePreview (or something similar) to copy the window to a bitmap/texture? (and relay inputs back & forth too I presume)