search

microsoft / MixedRealityToolkit-Unity

This repository is for the legacy Mixed Reality Toolkit (MRTK) v2. For the latest version of the MRTK please visit https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity
https://aka.ms/mrtkdocs
MIT License
6k stars 2.12k forks source link

WebGL build #7673

Closed Rufus31415 closed 9 months ago

Rufus31415 commented 4 years ago

Hello,

Describe the problem

Many contributors and app testers do not have a mixed reality device, such as a Hololens 2.

Describe the solution you'd like

Would it be possible to allow building in WebGL? The goal is to have the same rendering in the browser as in the Unity editor. A user can then simulate a mixed reality device inside its browser. I manage to do so by modifying some asmdef files and replacing some "#if UNITY_EDITOR" by "#if UNITY_EDITOR || UNITY_WEBGL". I would like to have this natively integrated into the MRTK.

Additional context

With this feature, you could make scene demonstrations (manipulation handler, solvers, shaders, ...) in the browser without the user having to download Unity, download the MRTK and read the documentation. For developers like me, we could extend the number of beta users without Hololens 2 required.

keveleigh commented 4 years ago

Definitely an interesting idea! What did you find that you needed to change from #if UNITY_EDITOR to #if UNITY_EDITOR || UNITY_WEBGL?

keveleigh commented 4 years ago

I think https://github.com/microsoft/MixedRealityToolkit-Unity/pull/7018 will help with this scenario as well, allowing for keyboard and mouse camera movement in builds outside the editor. Will ship in MRTK 2.4.

Rufus31415 commented 4 years ago

I didn't see #7018. Very interesting. I just tested the development branch and almost nothing is missing for the WebGL compilation to work!

I managed to compile for webgl very easily with the mrtk_development branch. For example, here is the HandInteractionExamples scene : https://rufus31415.github.io/sandbox/webgl-hololens2-simulation/

To achieve this, here are the changes I have made :

Audio

The following audio files have less than 256 samples, so they are not compatible with WebGL. The error is FSBTool ERROR: Failed encoding audio clip 'E:/MixedRealityToolkit-Unity/Assets/MRTK/SDK/StandardAssets/Audio/MRTK_Slider_Grab.wav' to AAC. Possibly the file is too short. Try to append silence such that the length becomes greater than 256 samples.

AR Foundation package

The namespace UnityEngine.SpatialTracking is unknown in file com.unity.xr.arfoundation@1.5.0-preview.6\Runtime\AR\ARSessionOrigin.cs I juste add some #if !UNITY_WEBGL around using and some code and then it was ok.

Weird

I get the following error in my Browser during a call to MixedRealityKeyboardPreview.ApplyShellSolverParameters() The virtual method SolverHandler.DetachFromCurrentTrackedObject() seems to be unknown I'll find out what's going on...

Invalid function pointer called with signature 'vii'. Perhaps this is an invalid value (e.g. caused by calling a virtual method on a NULL pointer)? Or calling a function with an incorrect type, which will fail? (it is worth building your source files with -Werror (warnings are errors), as warnings can indicate undefined behavior which can cause this)
An error occurred running the Unity content on this page. See your browser JavaScript console for more info. The error was:
uncaught exception: abort(158) at jsStackTrace (webgl3.asm.framework.unityweb:694:12)
stackTrace (webgl3.asm.framework.unityweb:708:11)
onAbort@http://10.34.0.23:8080/Build/UnityLoader.js:1039:50
abort (webgl3.asm.framework.unityweb:23116:10)
nullFunc_vii (webgl3.asm.framework.unityweb:16078:7)
b158 (webgl3.asm.code.unityweb:4017516:1)
__ZN18VirtActionInvoker06InvokeEjP12Il2CppObject (webgl3.asm.code.unityweb:3724186:1)
_SolverHandler_RefreshTrackedObject_m3A5EB1B08125E45C04CF98BEE5C9BA2DBE6BD4FA (webgl3.asm.code.unityweb:3737341:1)
_SolverHandler_get_TransformTarget_m3E1932AF88F9C115AC6B681B049866D2CE1F2361 (webgl3.asm.code.unityweb:2893668:1)
_Follow_get_ReferencePosition_m3D3137D6B03AF0E1EDC5D1F138EBE05F7BF80489 (webgl3.asm.code.unityweb:2467189:1)
_Follow_GetReferenceInfo_mEABA58344D33C9760D08C5451A793CE2B058F06B (webgl3.asm.code.unityweb:1229151:1)
_Follow_SolverUpdate_m39B0952412B366B1CD9FE742986198FDFC277F0D (webgl3.asm.code.unityweb:1053440:1)
__ZN18VirtActionInvoker06InvokeEjP12Il2CppObject (webgl3.asm.code.unityweb:3724186:1)
_MixedRealityKeyboardPreview_ApplyShellSolverParameters_mAC71E59553175615C59538E441D4935FE3AA955A (webgl3.asm.code.unityweb:1266045:1)

Mixed Reality Camera Setting Profile

To avoid a warning, the file Assets\MixedRealityToolkit.Providers\WindowsMixedReality\Shared\Microsoft.MixedReality.Toolkit.Providers.WindowsMixedReality.Shared.asmdef should include plateform WebGL. Related to #7119 and #7669

keveleigh commented 4 years ago

@Rufus31415 That's awesome! Thanks a ton for that link. Pretty cool to see things running in the browser!

Unfortunate about the ARFoundation package having an issue, but MRTK should work fine in this scenario with that package uninstalled.

Not sure what's going on under the Weird section though! Definitely weird...

@cre8ivepark Do you know if it might be possible to increase the samples in those audio clips to 256 without affecting other platforms?

Rufus31415 commented 3 years ago

@keveleigh With MRTK 2.5, all these problems no longer exist. It is possible to compile most of the example scenes in WebGL. I did it for several scenes here :

https://rufus31415.github.io/webxr/MRTK-HandInteraction/

You could offer WebGL examples directly on your website, right next to the documentation. Everyone could test the new features of MRTK without necessarily installing Unity.

I use this trick a lot so that people who don't have a Hololens can still test features or UX

Zee2 commented 3 years ago

@Rufus31415 Holy COW, this is super super super cool.

I noticed you chose my elastic menu/manipulator demo scene for one of your WebGL examples. It works really well on my end. That is super awesome.

@CDiaz-MS @wiwei @julenka This is really amazing.

Edit: seems like one or two of the elastic examples actually break with this build... Interesting, I wonder if it is a WebGL-compilation-specific issue or if something actually broke in mainline.

Rufus31415 commented 3 years ago

Glad you like it ! Yes there would be some fine-tuning for WebGL

You can scroll horizontally the menu and here is the list of all scenes I tried :

This is also related to #8472, If you visit with a WebXR-compatible browser, the scene can become immersive.

IssueSyncBot commented 9 months ago

We appreciate your feedback and thank you for reporting this issue.

Microsoft Mixed Reality Toolkit version 2 (MRTK2) is currently in limited support. This means that Microsoft is only fixing high priority security issues. Unfortunately, this issue does not meet the necessary priority and will be closed. If you strongly feel that this issue deserves more attention, please open a new issue and explain why it is important.

Microsoft recommends that all new HoloLens 2 Unity applications use MRTK3 instead of MRTK2.

Please note that MRTK3 was released in August 2023. It features an all new architecture for developing rich mixed reality experiences and has a minimum requirement of Unity 2021.3 LTS. For more information about MRTK3, please visithttps://www.mixedrealitytoolkit.org.

Thank you for your continued support of the Mixed Reality Toolkit!