Running Unity embedded in a subview (iOS)

[SGoerzen]

Hello,

I very like MAUI and your approach. Thanks for that!

It would be nice, if Unity can be somehow embedded in a subview instead of fullscreen. This way it would be easy to overlay MAUI controls over the application. I have seen many similar approach in other stacks (native ObjC, native Swift, ReactNative, Flutter).

I thought about to add another runEmbedded function and map it in maui by providing a UIView.

I am C# expert, but not ObjC or MAUI. So the scripts are not maybe not the correct way, but a first try for achieving this.

Unity project:

Classes/main.mm

- (void)runEmbeddedWithArgc:(int)argc argv:(char*[])argv appLaunchOpts:(NSDictionary*)appLaunchOpts inView:(UIView*)parentView
{
    if (self->runCount)
    {
        // initialize from partial unload ( sceneLessMode & onPause )
        UnityLoadApplicationFromSceneLessState();
        UnitySuppressPauseMessage();
        [self pause: false];
        [self showUnityWindow];

        // Ensure Unity view is a subview of the provided parent view
        UIView *unityView = UnityGetGLView();
        if (unityView.superview != parentView) {
            [unityView removeFromSuperview];
            [parentView addSubview:unityView];
            unityView.frame = parentView.bounds;
            unityView.autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;
        }

        // Send Unity start event
        UnitySendEmbeddedLaunchEvent(0);
    }
    else
    {
        // full initialization from ground up
        [self frameworkWarmup: argc argv: argv];

        id app = [UIApplication sharedApplication];

        id appCtrl = [[NSClassFromString([NSString stringWithUTF8String: AppControllerClassName]) alloc] init];
        [appCtrl application: app didFinishLaunchingWithOptions: appLaunchOpts];

        [appCtrl applicationWillEnterForeground: app];
        [appCtrl applicationDidBecomeActive: app];

        // Ensure Unity view is a subview of the provided parent view
        UIView *unityView = UnityGetGLView();
        [parentView addSubview:unityView];
        unityView.frame = parentView.bounds;
        unityView.autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;

        // Send Unity start (first time) event
        UnitySendEmbeddedLaunchEvent(1);
    }

    self->runCount += 1;
}

add in UnityFramework.h - (void)runEmbeddedWithArgc:(int)argc argv:(char*[])argv appLaunchOpts:(NSDictionary*)appLaunchOpts inView:(UIView*)parentView;

iOSBridge Project:

ApiDefinitions.cs (interface UnityFramework)

 [Export("runEmbeddedWithArgc:argv:appLaunchOpts:parentView:")]
        void RunEmbeddedWithArgc(int argc, IntPtr argv, NSDictionary options, UIView parentView);

MAUI Project:

Extend UnityBridge.iOS.cs by following

private static void InitialiseUnity(UIKit.UIView parentView = null)
        {
            if (IsUnityInitialised)
            {
                return;
            }

#if __IOS__ 
            framework = UnityFramework.LoadUnity();

            framework.RegisterFrameworkListener(new UnityBridge_UnityFrameworkListener());
            Bridge.RegisterUnityContentReceiver(new UnityBridge_UnityContentReceiver());

            if (parentView == null)
            {
                framework.RunEmbedded();
            }
            else
            {
                var argc = 0;
                var argv = IntPtr.Zero;
                var options = new NSDictionary();
                framework.RunEmbeddedWithArgc(argc, argv, options, parentView);
            }
#endif
        }

        public static void ShowUnityWindow(UIKit.UIView parentView = null)
        {
            if (!IsUnityInitialised)
            {
                InitialiseUnity(parentView);
            }
#if __IOS__ 
            if (framework != null)
            {
                framework.ShowUnityWindow();
            }
#endif
        }

and call this stuff from anywhere by UnityBridge.ShowUnityWindow(unityHost.ToPlatform(Handler.MauiContext));, where unityHost is a element.

Unfortunately, I am getting the error `ObjCRuntime.ObjCException: Objective-C exception thrown.  Name: NSInvalidArgumentException Reason: -[UnityFramework runEmbeddedWithArgc:argv:appLaunchOpts:parentView:]: unrecognized selector sent to instance 0x301e1f780
Native stack trace:
    0   CoreFoundation                      0x00000001a4decf2c

Do you have an idea how to solve this use case? @matthewrdev

[matthewrdev]

Hi @SGoerzen,

We have experimented with this at Red-Point, with a little bit of success, but haven't made a concerted effort to use MAUI over the top of Unity.

It is possible (somewhat) but we found it had a few caveats:

• As outlined the Unity As A Library docs, only full screen rendering is supported. Therefore, nesting Unity into a MAUI control within a page is not feasible as it effectively breaks the Unity player.

• Rather than injecting the view through your changes above, you can use native embedding (your usage of ToPlatform) and inject the UIView into the framework.AppController().RootView.

  This is a sample of how we were experimenting with it:

          var mauiContext = new MauiContext(App.ServiceProvider);
          overlay = new UnityPlayerOverlay();
          overlay.Handler ??= overlay.ToHandler(mauiContext);
  
          var overlayView = overlay.Handler.PlatformView as UIKit.UIView;
  
          var controller = framework.AppController();
          controller.RootView.AddSubview(overlayView);
          controller.RootView.BringSubviewToFront(overlayView);
  
          overlayView.Frame = controller.RootView.Frame;
          overlayView.AutoresizingMask = UIViewAutoresizing.FlexibleWidth | UIViewAutoresizing.FlexibleHeight;
  
          overlayView.SetNeedsDisplay();

  We found it worked (somewhat) but it caused issues with touch events inside the 3D player. At this point in time, we won't be pursuing it further.

  Hopefully that answers your question!

[SGoerzen]

@matthewrdev Thank you for reply! Sad to hear but it helps a lot.

Do you know if it has the same issues on other platforms? (As Unity has mentioned it on their page) Or is it just a MAUI problem?

[matthewrdev]

@SGoerzen I would image this affects other platforms as Unity explicitly mentions it in their documentation.

To use native controls for Unity, the approach is to host them inside the Unity root view, rather than hosting the Unity viewer inside the native control.

Best of luck with the project!

[SGoerzen]

@matthewrdev thanks for your quick answers!

tried out the Flutter widget and integrated my AR application into the Flutter app with working gesture control (Vuforia is still to be tested).

I really dislike Flutter... but I guess I need to use it for now.

For long term: What do you think, would it be possible to adapt this widget for MAUI? I think there is needed to adapt AppDelegate.cs and e.g. translate especially these classes into C# and MAUI: https://github.com/juicycleff/flutter-unity-view-widget/tree/master/ios/Classes

Is it worth trying it?