iOS Problem on ARFoundation With the use of plane detection and image tracking

[marcomameli1992]

Dear, I'm trying to use the Image target and pose the object with plane detection. Yesterday it works but today does not work. I have these messages: `2020-07-10 20:33:57.212573+0200 AR-Contents[739:72852] Built from '2019.3/staging' branch, Version '2019.3.15f1 (59ff3e03856d)', Build type 'Release', Scripting Backend 'il2cpp' -> applicationDidFinishLaunching() -> applicationDidBecomeActive() [Subsystems] Discovering subsystems at path /private/var/containers/Bundle/Application/DF1F8698-F4E7-4E27-B8B6-5ACDCBAB75F7/AR-Contents.app/Data/UnitySubsystems [Subsystems] No descriptors matched for examples in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json. [Subsystems] 1 'inputs' descriptors matched in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json [Subsystems] No descriptors matched for displays in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json. [Subsystems] 1 'meshings' descriptors matched in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json GfxDevice: creating device client; threaded=1 Initializing Metal device caps: Apple A12 GPU Initialize engine version: 2019.3.15f1 (59ff3e03856d) XRGeneral Settings awakening... UnityEngine.XR.Management.XRGeneralSettings:Awake()

(Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 35)

2020-07-10 20:33:57.488791+0200 AR-Contents[739:72852] // 2D joint skeleton enum JointIndices { Invalid = -1, Head = 0, // parent: Neck1 [1] Neck1 = 1, // parent: Root [16] RightShoulder1 = 2, // parent: Neck1 [1] RightForearm = 3, // parent: RightShoulder1 [2] RightHand = 4, // parent: RightForearm [3] LeftShoulder1 = 5, // parent: Neck1 [1] LeftForearm = 6, // parent: LeftShoulder1 [5] LeftHand = 7, // parent: LeftForearm [6] RightUpLeg = 8, // parent: Root [16] RightLeg = 9, // parent: RightUpLeg [8] RightFoot = 10, // parent: RightLeg [9] LeftUpLeg = 11, // parent: Root [16] LeftLeg = 12, // parent: LeftUpLeg [11] LeftFoot = 13, // parent: LeftLeg [12] RightEye = 14, // parent: Head [0] LeftEye = 15, // parent: Head [0] Root = 16, // parent: [-1] } 2020-07-10 20:33:57.495996+0200 AR-Contents[739:72852] // 3D joint skeleton enum JointIndices { Invalid = -1, Root = 0, // parent: [-1] Hips = 1, // parent: Root [0] LeftUpLeg = 2, // parent: Hips [1] LeftLeg = 3, // parent: LeftUpLeg [2] LeftFoot = 4, // parent: LeftLeg [3] LeftToes = 5, // parent: LeftFoot [4] LeftToesEnd = 6, // parent: LeftToes [5] RightUpLeg = 7, // parent: Hips [1] RightLeg = 8, // parent: RightUpLeg [7] RightFoot = 9, // parent: RightLeg [8] RightToes = 10, // parent: RightFoot [9] RightToesEnd = 11, // parent: RightToes [10] Spine1 = 12, // parent: Hips [1] Spine2 = 13, // parent: Spine1 [12] Spine3 = 14, // parent: Spine2 [13] Spine4 = 15, // parent: Spine3 [14] Spine5 = 16, // parent: Spine4 [15] Spine6 = 17, // parent: Spine5 [16] Spine7 = 18, // parent: Spine6 [17] LeftShoulder1 = 19, // parent: Spine7 [18] LeftArm = 20, // parent: LeftShoulder1 [19] LeftForearm = 21, // parent: LeftArm [20] LeftHand = 22, // parent: LeftForearm [21] LeftHandIndexStart = 23, // parent: LeftHand [22] LeftHandIndex1 = 24, // parent: LeftHandIndexStart [23] LeftHandIndex2 = 25, // parent: LeftHandIndex1 [24] LeftHandIndex3 = 26, // parent: LeftHandIndex2 [25] LeftHandIndexEnd = 27, // parent: LeftHandIndex3 [26] LeftHandMidStart = 28, // parent: LeftHand [22] LeftHandMid1 = 29, // parent: LeftHandMidStart [28] LeftHandMid2 = 30, // parent: LeftHandMid1 [29] LeftHandMid3 = 31, // parent: LeftHandMid2 [30] LeftHandMidEnd = 32, // parent: LeftHandMid3 [31] LeftHandPinkyStart = 33, // parent: LeftHand [22] LeftHandPinky1 = 34, // parent: LeftHandPinkyStart [33] LeftHandPinky2 = 35, // parent: LeftHandPinky1 [34] LeftHandPinky3 = 36, // parent: LeftHandPinky2 [35] LeftHandPinkyEnd = 37, // parent: LeftHandPinky3 [36] LeftHandRingStart = 38, // parent: LeftHand [22] LeftHandRing1 = 39, // parent: LeftHandRingStart [38] LeftHandRing2 = 40, // parent: LeftHandRing1 [39] LeftHandRing3 = 41, // parent: LeftHandRing2 [40] LeftHandRingEnd = 42, // parent: LeftHandRing3 [41] LeftHandThumbStart = 43, // parent: LeftHand [22] LeftHandThumb1 = 44, // parent: LeftHandThumbStart [43] LeftHandThumb2 = 45, // parent: LeftHandThumb1 [44] LeftHandThumbEnd = 46, // parent: LeftHandThumb2 [45] Neck1 = 47, // parent: Spine7 [18] Neck2 = 48, // parent: Neck1 [47] Neck3 = 49, // parent: Neck2 [48] Neck4 = 50, // parent: Neck3 [49] Head = 51, // parent: Neck4 [50] Jaw = 52, // parent: Head [51] Chin = 53, // parent: Jaw [52] LeftEye = 54, // parent: Head [51] LeftEyeLowerLid = 55, // parent: LeftEye [54] LeftEyeUpperLid = 56, // parent: LeftEye [54] LeftEyeball = 57, // parent: LeftEye [54] Nose = 58, // parent: Head [51] RightEye = 59, // parent: Head [51] RightEyeLowerLid = 60, // parent: RightEye [59] RightEyeUpperLid = 61, // parent: RightEye [59] RightEyeball = 62, // parent: RightEye [59] RightShoulder1 = 63, // parent: Spine7 [18] RightArm = 64, // parent: RightShoulder1 [63] RightForearm = 65, // parent: RightArm [64] RightHand = 66, // parent: RightForearm [65] RightHandIndexStart = 67, // parent: RightHand [66] RightHandIndex1 = 68, // parent: RightHandIndexStart [67] RightHandIndex2 = 69, // parent: RightHandIndex1 [68] RightHandIndex3 = 70, // parent: RightHandIndex2 [69] RightHandIndexEnd = 71, // parent: RightHandIndex3 [70] RightHandMidStart = 72, // parent: RightHand [66] RightHandMid1 = 73, // parent: RightHandMidStart [72] RightHandMid2 = 74, // parent: RightHandMid1 [73] RightHandMid3 = 75, // parent: RightHandMid2 [74] RightHandMidEnd = 76, // parent: RightHandMid3 [75] RightHandPinkyStart = 77, // parent: RightHand [66] RightHandPinky1 = 78, // parent: RightHandPinkyStart [77] RightHandPinky2 = 79, // parent: RightHandPinky1 [78] RightHandPinky3 = 80, // parent: RightHandPinky2 [79] RightHandPinkyEnd = 81, // parent: RightHandPinky3 [80] RightHandRingStart = 82, // parent: RightHand [66] RightHandRing1 = 83, // parent: RightHandRingStart [82] RightHandRing2 = 84, // parent: RightHandRing1 [83] RightHandRing3 = 85, // parent: RightHandRing2 [84] RightHandRingEnd = 86, // parent: RightHandRing3 [85] RightHandThumbStart = 87, // parent: RightHand [66] RightHandThumb1 = 88, // parent: RightHandThumbStart [87] RightHandThumb2 = 89, // parent: RightHandThumb1 [88] RightHandThumbEnd = 90, // parent: RightHandThumb2 [89] } [Subsystems] Loading plugin UnityARKit for subsystem ARKit-Input... [Subsystems] UnityARKit successfully registered Provider for ARKit-Input [Subsystems] Loading plugin UnityARKit for subsystem ARKit-Meshing... [Subsystems] Failed to initialize subsystem ARKit-Meshing [error: 1] 2020-07-10 20:33:57.680809+0200 AR-Contents[739:72852] Unbalanced calls to begin/end appearance transitions for <UIViewController: 0x125a6e070>. UnloadTime: 1.936042 ms NullReferenceException: Object reference not set to an instance of an object. at ProvaScript.OnEnable () [0x00000] in <00000000000000000000000000000000>:0

(Filename: currently not available on il2cpp Line: -1)

NullReferenceException: Object reference not set to an instance of an object. at ProvaScript.ShowTrackerInfo () [0x00000] in <00000000000000000000000000000000>:0 at ProvaScript.Start () [0x00000] in <00000000000000000000000000000000>:0

(Filename: currently not available on il2cpp Line: -1)

2020-07-10 20:33:59.677075+0200 AR-Contents[739:72852] UnityARKit: Updating ARSession configuration with <ARWorldTrackingConfiguration: 0x2812be1c0 worldAlignment=Gravity lightEstimation=Disabled frameSemantics=None videoFormat=<ARVideoFormat: 0x2804d0320 imageResolution=(1920, 1440) framesPerSecond=(60)> autoFocus=Enabled environmentTexturing=None wantsHDREnvironmentTextures=Enabled planeDetection=Horizontal|Vertical collaboration=Disabled userFaceTracking=Disabled sceneReconstruction=None>`

[tdmowrer]

You have unhandled exceptions in something called ProvaScript, which is not part of ARFoundation:

NullReferenceException: Object reference not set to an instance of an object.
at ProvaScript.OnEnable () [0x00000] in <00000000000000000000000000000000>:0

(Filename: currently not available on il2cpp Line: -1)

NullReferenceException: Object reference not set to an instance of an object.
at ProvaScript.ShowTrackerInfo () [0x00000] in <00000000000000000000000000000000>:0
at ProvaScript.Start () [0x00000] in <00000000000000000000000000000000>:0

(Filename: currently not available on il2cpp Line: -1)

[marcomameli1992]

I have inserted this script like a component of ARSessionOrigion and yesterday it works and now it is in the same position and now it does not work

[tdmowrer]

The problem is in your script. OnEnable and ShowTrackerInfo are both throwing an exception. Look for what could be null, and maybe add some Debug.Logs.

[marcomameli1992]

With debug .log I found that the ARPlaneManager is null but I do not understand why.

[tdmowrer]

Sorry, I don't have enough context to help. Can you share the body of that method?

[marcomameli1992]

I can share all the script code. The object where it is used as a component is the ARSessionOrigin where there is the ARPlaneManager component

using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.Serialization;
using UnityEngine.XR.ARFoundation;
using UnityEngine.XR.ARSubsystems;

public class TrackImageRuntimeManager : MonoBehaviour
{
    public delegate void ImageTarget();
    public static event ImageTarget OnImageTarget;

    [SerializeField] private GameObject prefabObject;

    private ARTrackedImageManager _arTrackedImageManager;
    private ARSessionOrigin _arSessionOrigin;
    private ARPlaneManager _arPlaneManager;
    private ARPlane _arPlane = null;

    // Start is called before the first frame update
    void Start()
    {
        _arTrackedImageManager = gameObject.AddComponent<ARTrackedImageManager>();
        _arSessionOrigin = GetComponent<ARSessionOrigin>();
        _arPlaneManager = GetComponent<ARPlaneManager>();

        _arTrackedImageManager.referenceLibrary = _arTrackedImageManager.CreateRuntimeLibrary();
        _arTrackedImageManager.requestedMaxNumberOfMovingImages = 3;

        _arTrackedImageManager.enabled = true;

        _arTrackedImageManager.trackedImagesChanged += OnTrackedImagesChanged;
    }

    private IEnumerator CaptureImage()
    {
        yield return new WaitForEndOfFrame();

        var texture = ScreenCapture.CaptureScreenshotAsTexture();

        StartCoroutine(AddImageJob(texture));
    }

    public IEnumerator AddImageJob(Texture2D texture2D)
    {
        yield return null;

        var firstGuid = new SerializableGuid(0,0);
        var secondGuid = new SerializableGuid(0,0);

        XRReferenceImage newImage = new XRReferenceImage(firstGuid, secondGuid, 
            new Vector2(0.1f,0.1f), Guid.NewGuid().ToString(), texture2D);

        try
        {
            MutableRuntimeReferenceImageLibrary mutableRuntimeReferenceImageLibrary = 
                _arTrackedImageManager.referenceLibrary as MutableRuntimeReferenceImageLibrary;

            var jobHandle = mutableRuntimeReferenceImageLibrary.ScheduleAddImageJob(texture2D, 
                Guid.NewGuid().ToString(), 1f);

            while(!jobHandle.IsCompleted)
            {
                continue;
            }
        }
        catch(Exception)
        {
            if(texture2D is null)
            {
                Debug.Log("TrackImage: Null image obtained"); 
            }
        }
    }

    private void PlanesChanged(ARPlanesChangedEventArgs arPlanesChangedEventArgs)
    {
        if (arPlanesChangedEventArgs.added != null)
        {
            _arPlane = arPlanesChangedEventArgs.added[0];
        }
    }

    private void OnDisable()
    {
        _arTrackedImageManager.trackedImagesChanged -= OnTrackedImagesChanged;
    }

    void OnTrackedImagesChanged(ARTrackedImagesChangedEventArgs eventArgs)
    {
        foreach (ARTrackedImage trackedImage in eventArgs.added)
        {
            Debug.Log("Immagine aggiunta, creo l'oggetto " + prefabObject.name);

            if (_arPlane != null)
            {
                GameObject added = Instantiate(prefabObject, _arPlane.transform.position, Quaternion.identity);
                added.SetActive(true);
            }
            else
            {
                Debug.Log("ARPlane null");
            }

            OnImageTarget?.Invoke();
        }

        foreach (ARTrackedImage trackedImage in eventArgs.updated)
        {

        }
    }

    private void OnEnable()
    {
        Debug.Log("On Enable di TrackImage: ");
        if (_arPlaneManager is null)
        {
            Debug.Log("AR Plane Manager is null");
        }
        else
        {
            _arPlaneManager.planesChanged += PlanesChanged;
        }

        GameManager.OnGetImageEvent += () => StartCoroutine(CaptureImage());

    }
}

[tdmowrer]

OnEnable is called before Start (see this chart), so _arPlaneManager will be null when OnEnable is executed for the first time.

[marcomameli1992]

Thank you for your help!!

Can I ask you another question: If I define some anchor points is possible to use it as a bounding box to place an object?

[tdmowrer]

I don't think I understand the question, but anchors don't sound like the right solution. If you just want to instantiate an object at the center of a bounding box defined by points (feature points?) you could just average them.

[marcomameli1992]

Thank you for your help. Now I'm opening a new question for how-to information.