Unity Error DllNotFoundException: cardboard_api

[laura2105]

Hi there, I'm getting an error that keeps spamming my console as soon as Aryzon mode is enabled:

DllNotFoundException: cardboard_api assembly:<unknown assembly> type:<unknown type> member:(null)
at (wrapper managed-to-native) Aryzon.AryzonMainPoseDriver.CardboardUnity_AddSixDoFData(intptr,long,single[],single[])
at Aryzon.AryzonMainPoseDriver.AddSixDoFData (UnityEngine.Vector3 position, UnityEngine.Quaternion rotation, System.Int64 timestampNs) [0x0004e] in C:\Users\laura\MA-AR-Exploration-Rubikscube\Library\PackageCache\com.aryzon.unity-sdk@dc20305ed2\Runtime\Scripts\AryzonMainPoseDriver.cs:160 
at Aryzon.AryzonMainPoseDriver.ArCameraManager_frameReceived (UnityEngine.XR.ARFoundation.ARCameraFrameEventArgs obj) [0x000d1] in C:\Users\laura\MA-AR-Exploration-Rubikscube\Library\PackageCache\com.aryzon.unity-sdk@dc20305ed2\Runtime\Scripts\AryzonMainPoseDriver.cs:186 
at UnityEngine.XR.ARFoundation.ARCameraManager.InvokeFrameReceivedEvent (UnityEngine.XR.ARSubsystems.XRCameraFrame frame) [0x003ab] in C:\Users\laura\MA-AR-Exploration-Rubikscube\Library\PackageCache\com.unity.xr.arfoundation@4.2.7\Run

Apparently the system keeps searching for a dll-file with name cardboard_api that can't be found. I discovered it in my own project and also tried out your sample scene - same issue. The dll-File is nowhere to be found. I followed your instructions in the readme regarding the setup.

I'm using Unity 2021.3.16f1 on Windows 11 with AR Core. I build for Android Minimum API Level 25, Target API Level 33.

[maartenslaa]

Hi Laura,

In the PackageManager can you check if you have both these packages installed? Also make sure your target platform is Android or iOS.

Regarding the second package, use our fork from the Google Cardboard XR Plugin, not the one from Google: https://github.com/Aryzon/cardboard-xr-plugin.git

Hope this helps!

[laura2105]

Hi Maarten, thanks for the quick reply!

Yes, all of these packages installed, updated to most recent version and target platform is set to Android.

Maybe I need to clarify: The build itself works, I can get the app on my phone to run. But once running the build, this error keeps popping up every frame. Since I'm not sure how the SDK's 6DoF is supposed to look like WITHOUT this error, I can't confirm that this error is to be ignored and doesn't spoil anything. It's my first time trying out your SDK and I don't really know whether to ignore this error or if some functionality isn't working because of it.

[maartenslaa]

I see what happened, the constant had changed and we didn't test well enough on Android. I pushed a fix, can you try updating the Aryzon MR Headset Unity SDK? Simply hit update in the package manager.

[laura2105]

Awesome, thanks, that fixed the console error!

However, my 3D models I placed in AR are now flickering terribly and I see every model twice. Before, they were displaying just fine. I tried to screenshot it, but on the screenshot it looked correct... Could that be something to do with your fix? Maybe an issue with my framerate could explain the flickering, but I don't get why the model appears twice, rendered (and flickering) next to each other. Thanks in advance!

[maartenslaa]

Could be.. I will get a closer look later today. For now, on the ARSession component, can you toggle Match Frame Rate to the value it was not?

Do you also see double when the phone is not moving? Does this happen in normal (non-stereoscopic) AR mode as well?

For testing without the flickering you may want to try rotational only tracking until there is a fix for this.

It can also be that performance mode is not turned off correctly.

I am sorry for the trouble you are having!

[laura2105]

No problem, thanks for your quick replies!

I activated "Match Frame Rate" now for the second try and it looks much better! Previously I had it disabled as suggested for ARFoundation. Now the 3D model just appears double and flickers sometimes when I move the phone, but not when holding still.

I also tried with the Tracking Mode of the ARSession component both set to "Position and Rotation" and "Rotation only" and it doesn't really make a difference. When moving the phone, the model sometimes appears twice and flickers or even jumps to another position. Hope this is understandable.

The non-stereoscopic AR mode was fine btw, it only occurs once starting the Aryzon mode. Thanks in advance!

EDIT: Ignore, what I wrote before, I thoroughly tested the flickering - it is the same for both "Position and Rotation" and "Rotation only" set as tracking mode. The model first seems fine for like 2 seconds, then it starts flickering and after like 5 seconds flickering it suddenly kind of refreshes and doesn't flicker anymore. Afterwards the cycle begins again, with 2 seconds of not-flickering, then again flickering etc... This flickering cycle keeps happening all the time, regardless of whether I'm moving the phone, moving the marker or holding still. The flickering cycle also only occurs in Aryzon mode (stereoscopic view). In non-stereoscopic view, the model looks completely fine.

[laura2105]

Hi, any updates on the flickering and jumping of the model? I kinda need this working by the end of this week for a project and haven't been able to find good settings to fix this. It only occurs in stereoscopic Aryzon mode and not with simple ARFoundation (ARCore).

[maartenslaa]

Hi, it is a bit hard to debug for me since I have not been able to replicate the issue here. Can you change AryzonMainPoseDriver.cs by commenting out line 160?

//CardboardUnity_AddSixDoFData(AryzonCardboardSubsystemLoader.inputPointer, timestampNs, positionArray, rotationArray);

Then can you try this?

rotation = Quaternion.identity;
float[] rotationArray = { rotation.x, rotation.y, rotation.z, rotation.w };
CardboardUnity_AddSixDoFData(AryzonCardboardSubsystemLoader.inputPointer, timestampNs, positionArray, rotationArray);

I'm curious when the flickering occurs.

What phone are you using? Do you have access to another phone?

[laura2105]

Hi, thanks for your reply, I understand!

I might have to add an important detail, sorry that I didn't mention that earlier... The whole jumping-to-positions-thing is occurring when tracking an image with ARFoundation! For example your demo interaction 6DoF scene works fine on my phone, but there is no image tracking in it.

I've tried to test as many variants for you as I can. :-) For reference: I'm using a Samsung Galaxy S21 FE 5G with Android 13. I might be able to borrow a friend's phone to test the app on another device, yes.

This is the status quo without any of your suggestions applied: Match Frame Rate Activated + Rotational tracking Only = Tracked model keeps flickering and jumping to new positions in certain time intervals Match Frame Rate Activated + Position and rotational tracking mode = Tracked model keeps flickering and jumping to new positions in certain time intervals Match Frame Rate Deactivated + Rotational tracking Only = Tracked model flickers, but position stays correct and constant Match Frame Rate Deactivated + Position and rotational tracking mode = Tracked model flickers and appears twice and jumps to new positions.

These are the results when commenting out the 6DoF line: Match Frame Rate Deactivated + Rotational tracking Only = image tracking doesn't work anymore in Aryzon mode, model doesn't appear Match Frame Rate Deactivated + Position and Rotational tracking mode = image tracking doesn't work anymore in Aryzon mode, model doesn't appear Match Frame Rate Activated + Position and Rotational tracking mode = image tracking doesn't work anymore in Aryzon mode, model doesn't appear Match Frame Rate Activated + Rotational tracking Only = No flickering or jumping at all! But model isn't being rendered at the correct position on top of the image, but instead to the right side and a little below it.

These are the results when adding Quaternion.identity: Match Frame Rate Deactivated + Position and Rotational tracking mode = image tracking doesn't work anymore in Aryzon mode, model doesn't appear Match Frame Rate Deactivated + rotational tracking only = model is being rendered at the right side of the screen, almost out of sight. It keeps flickering terribly but doesn't really jump to a different position. Match Frame Rate Activated + Rotational tracking Only = Tracked model keeps flickering and jumping to new positions in certain time intervals Match Frame Rate Activated + Position and Rotational tracking mode = model is being rendered in bottom right corner of the screen and keeps drifting out of sight when adjusting the phone.

All variants where the image tracking doesn't seem to "work anymore" could also be just rendering the model out of sight. The image was detected in all of the variants, I just wasn't able to see the model that should be rendered on top of it.

So to make it short: The best result regarding the model holding its position on the tracked image I'm getting with rotational tracking only and the "match frame rate" option deactivated. It just keeps flickering (because of the frame rate?), otherwise it would be working great! Also, commenting the 6DoF line out, activating "match frame rate" and "rotational tracking only" got completely rid of flickering or position jumping! The model is just rendered at the wrong position.

Hope my testing helps.

[maartenslaa]

Hey thanks for testing so throughly, this helps me understand a bit better. A couple more questions:

1. Is it only the model connected to the marker that flickers, or other models placed at a fixed position in the scene as well?
2. Try this: scan the marker so the object appears, move the phone so the marker is out of view, take away the marker, move the phone back so it 'sees' the position where the marker should have been. Does the model still flicker?

This is without the changes to AryzonMainPoseDriver.

[laura2105]

No problem!

No, unfortunately other models flicker as well. I tried with "Rotation only", "positional and rotational tracking" and deactivated "match frame rate". Terrible flickering for both models. When activating the "match frame rate" option, the flickering is much slower (if that makes sense), the models look fine for slightly longer periods of time.

For 2. I don't get the model appearing where it "should be" without the marker. The model only appears if the marker is in sight. When I remove the marker, the model simply doesn't render again... I'm attaching an anchor to the tracked image, not the world space, I believe.

[maartenslaa]

I find 2 a bit strange, what exactly are you using for marker tracking? If possible, can you share your project?

[laura2105]

Sharing the whole project probably won't work, but I'm using this script for the image tracking and didn't really modify anything relevant. It's my first project using ARFoundation, that's why I don't really know any better... I'd of course appreciate any tips! Before, I've only used Vuforia for image tracking.

Hope that helps :-)

[maartenslaa]

I see, set this:

.SetActive(trackedImage.trackingState == TrackingState.Tracking);

to

.SetActive(true);

Then do '2.' again where you remove the marker. Does it still flicker when the marker is removed?

[laura2105]

Yeah, just commenting out that line helped, too :-) Now the model instance stays where it was tracked first. However, it still seems fine at first, and then the phone starts trying to adjust the position. As I described earlier, the model looks fine for like 3 seconds, then it starts to disappear and reappear in continuously shorter time periods (looks like slowly increasing flickering) until it finally disappears completely. Then after another 2 seconds it reappears at its initial position again and the cycle starts another time. My settings still are "Match frame rate" activated and "Position and rotational tracking".

Note: for rotational tracking only with activated "match frame rate", the model doesn't disappear completely at the "end of the cycle" - the phone just keeps adjusting the position in frequent time intervals, so that the model hops to different positions like every two seconds. Is it supposed to be like that? My phone seems to want to adjust the position the whole time.

[maartenslaa]

Sorry about all this trial and error stuff.. I have created 2 .aar files. The first removes the smoothing of the rotation, maybe that helps? The second sets the ARFoundation rotation to the cardboard rotation which is slower to update but may work for you.

1. GfxPluginCardboard.zip
2. GfxPluginCardboard.zip

You can try them out by replacing the file I have selected here (do not remove the .meta file):

[laura2105]

No problem!

I'm not able to exchange the files in the package folders... Unity complains.

While I can catch you here - any hint on how I can show a screen overlay (canvas) in stereoscopic mode? Basically like the reticle (which isn't showing for me btw, only in your sample scene) or the line that separates the eyes. I couldn't find where for example the line or the settings button are being rendered, maybe you can point me to the right script so I can adapt for my own Text overlay?

[maartenslaa]

You should be able to replace them outside of Unity in windows explorer. Just don't remove the .meta files, that's when Unity complains and puts back the original file.

I believe you can simply add a screen overlay canvas but you will need to disable the 'Disable Screen Overlays In Stereoscopic Mode' option on the AryzonInputController GameObject.

[laura2105]

Still not sure if that worked. I renamed the old file and then copied the new one in. Not sure if Unity can be fooled. The package cache was invalidated and rebuilt because the following immutable asset(s) were unexpectedly altered: Packages/com.google.xr.cardboard/Runtime/Android/GfxPluginCardboard.aar appears. I tested both .aar files and none made a difference... :-(

Yeah, already tried that for the canvas, but it doesn't work. The reticle shows up at start-up for like a millisecond and then vanishes and never reappears. I can't get the canvas items to render.

[maartenslaa]

The message appears for me as well, you can ignore it. I now believe the issue is not in the rotation part but in the positioning part of the code. What happens when you set the position to zero instead of the rotation in AryzonMainPoseDriver? Try not to move the phone too much after starting your app, only for putting it in landscape:

float[] positionArray = { 0f, 0f, 0f };

[laura2105]

Tried that, still flickering :(

[maartenslaa]

Hmm.. it's really hard to debug this when I don't have the project. It would really help if you can share an example project where the issue occurs.

[laura2105]

Ok, I'll try later today.

[laura2105]

Find an example project here: WeTransfer-Link To be honest, I for some reason couldn't get the image tracking working, must be something in the configurations I might have missed... Don't have enough time to debug it thoroughly. I added a sphere at a fixed position however and that flickers/jumps for me as well in this project. So should be helpful enough for you :-)

Regarding the canvas - I noticed the reticle and my canvas showing up when testing the Aryzon mode in Game mode in the Unity editor. In the editor, I'm also not able to see the stereoscopic Cardboard view as I do on my phone. However, when testing on my phone, the Cardboard view (the line between the two eye views, the exit and settings buttons) seems to block out any canvases that I added to the scene. Any ideas, what I could do to prevent this from happening and display the canvas correctly in stereoscopic mode? I already played around with the sorting layer of the canvas. What's interesting is, that the reticle only shows for me when I have your interaction example in the scene. Does the reticle need objects in the scene to be hit by the raycast for it to show? If yes, how can I imitate that behavior to have for my canvas at all times?

[maartenslaa]

Thanks for the project, I tried it out and for me it was again the same. No flickering. I did however see one small thing that could cause flickering which I changed. You can try this out by updating the cardboard-xr-plugin in your package manager.

Regarding your other questions: The editor cannot render the stereoscopic mode.

I'm not sure about the canvas overlay in stereoscopic mode, what is the reason for this? If it is meant to be seen in the headset, you should not use an overlay but instead a world canvas because you will need to render for both eyes with a certain distortion. The stereoscopic renderer does not render a screen overlay canvas. I'm not sure if you can accomplish to view a screen overlay canvas on top of the stereoscopic rendering. The buttons and line you see there are rendered by the cardboard sdk. These are not Unity objects. Have a look at the RotateWithCamera.cs script, perhaps this will help you here. It can rotate an object so it will stay in view, while still giving 'headroom' to rotate the head independently of the object.

The reticle only shows when first pointing towards a raycast object.

[laura2105]

Hi, thanks again for all your help - after the update it looks sooo much better! I can even switch to "positional and rotational tracking" and deactivate "match frame rate" (although the last one didn't really make any difference anymore). Thanks A LOT!

The canvas was thought for an instruction string appearing on screen that is always in sight, but I guess I'll stick to a world canvas then. I had discovered the RotateWithCamera.cs script already but couldn't really make sense of it. It doesn't work for me yet, but unless you have a brilliant tip for me I'll try figuring it out myself :-)

And thanks for the hint regarding the reticle. I probably won't use it anyway, just thought it might be some behavior I could copy for my instruction string with the onscreen canvas.

[maartenslaa]

I was pretty much out of ideas so I'm really glad this worked 😅

This is the setup for the RotateWithCamera script:

Parent GameObject with RotateWithCamera component x=y=z=0 --> Child GameObject with Canvas Transform x=y=0, z=1 (for instance, if the canvas is 1 meter away from the user)

Have a look at the Calibration view in the Aryzon GameObject for an example

[laura2105]

Ok thank you, will give that a go!