Camera rotation and App size

[KornyAS]

I have some problems with the plugin: First, It works fine on my linux, camera rotation is all right, but when I use my android I see that everything is 90 degree rotated, even the robot. Rotating a background plane has no effect, except my frontal camera (but the points and lines shows in incorrect positions, like camera image is still rotated). How to fix it?

Second, how can I decrease the size of application, I used stripping param when build, now it's 180 mb, but still too heavy. I wonder, how to decrease it gradually. I need only Instant motion tracking, nothing more. Maybe I can erase other functions somehow?

Third, it takes about 10 seconds to get android camera start but on linux it takes literally few milliseconds. How to decrease the starting time at least to 1-2 seconds?

[homuler]

First, It works fine on my linux, camera rotation is all right, but when I use my android I see that everything is 90 degree rotated, even the robot. Rotating a background plane has no effect, except my frontal camera (but the points and lines shows in incorrect positions, like camera image is still rotated). How to fix it?

Does it occur on FaceDetection graph, or does it occur only InstantMotionGraph?

I need only Instant motion tracking, nothing more. Maybe I can erase other functions somehow?

Please include only required calculators, models and assets and build it again. https://github.com/homuler/MediaPipeUnityPlugin/blob/master/mediapipe_api/BUILD#L110 https://github.com/homuler/MediaPipeUnityPlugin/blob/master/mediapipe_api/BUILD#L142

Third, it takes about 10 seconds to get android camera start but on linux it takes literally few milliseconds. How to decrease the starting time at least to 1-2 seconds?

In fact, it takes times to read model files from AssetBundle and save them to the device storage. By default, model files are read from AssetBundle, but you can place them under StreamingAssets and configure ResourceManager to read them. https://github.com/homuler/MediaPipeUnityPlugin/blob/master/Assets/Mediapipe/Samples/Scripts/SceneDirector.cs#L64

[KornyAS]

Does it occur on FaceDetection graph, or does it occur only InstantMotionGraph?

Both

[KornyAS]

I tried to erase everything except instant motion tracking, but application started to crash. I revert everything and after several experiments I find out that when we start Instant motion tracking as a first graph - application crashes but when I start another graph first and then I switch to Instant motion tracking- it works fine. Here are my crash logs: https://del.dog/ofignestaf.txt

BTW, I still don't know how to fix rotated camera

[KornyAS]

Please include only required calculators, models and assets and build it again.

I did exactly this

[homuler]

BTW, I still don't know how to fix rotated camera

Please see WebCamTexture#videoRotationAngle (though it may be a bug of sample apps).

I find out that when we start Instant motion tracking as a first graph - application crashes but when I start another graph first and then I switch to Instant motion tracking- it works fine.

I suspect this issue is caused by https://github.com/google/mediapipe/issues/1617 , which is not resolved yet.

Please include only required calculators, models and assets and build it again.

I did exactly this

Have you uninstalled all the resources first? If you've not removed files under Runtime/Resources, it does not take effect (unnecessary files will be still included in AssetBundle). I guess you can reduce the app size to around 100 ~ 110MB.

[KornyAS]

What the minimum of an app size? Could we reach at least 40-50?

Have you uninstalled all the resources first? If you've not removed files under Runtime/Resources, it does not take effect (unnecessary files will be still included in AssetBundle). I guess you can reduce the app size to around 100 ~ 110MB.

[homuler]

I don't know. I think it depends on the meaning of 'minimum' and your app.

On the plugin side, you can reduce the size of native libraries and model files to be included to the app. To reduce the size of native libraries, you can remove unused calculators or specify linker options as you mentioned. If you'd like to reduce more, please look up those linker options or compiler options.

To reduce the size of model files, simply include only required ones.

[KornyAS]

For instance, our app's size is 38 mb now, how much heavier will it be if we plug in a minimum of Instant motion tracking?

[Vladislav489]

Please, need your anwser @homuler

Например, размер нашего приложения сейчас составляет 38 МБ, насколько оно будет тяжелее, если мы подключим минимум мгновенного отслеживания движения?

[homuler]

Well, I don't know. Please see https://github.com/homuler/MediaPipeUnityPlugin/issues/131#issuecomment-829952648 and test by yourself.

[KornyAS]

Please see WebCamTexture#videoRotationAngle (though it may be a bug of sample apps)

Yes, it fixed camera rotation, but robot in instant motion tracking is still rotated, I don't know how to fix it

[KornyAS]

Could you please tell me, is it possible to create smt like this:

https://user-images.githubusercontent.com/37486099/117882871-72368d80-b2b3-11eb-8e41-5c05506e6aa3.mp4

First, you can move the circle on your table and then you can link it to a special point. After this you can open a model and it'll be downloaded and model will appear in the circle. The circle should be as a GameObject. If it's possible, could you tell me how to make it?

[KornyAS]

And I tried to find some sort of manual but I didn't find anything, maybe you know anything about it? @homuler

[Vladislav489]

@homuler We need to understand this, please answer

[emreboyraz0]

Could you please tell me, is it possible to create smt like this:

video_2021-05-11_20-48-30.mp4 First, you can move the circle on your table and then you can link it to a special point. After this you can open a model and it'll be downloaded and model will appear in the circle. The circle should be as a GameObject. If it's possible, could you tell me how to make it?

This plugin not doing AR exactly what you want. Please check out ARkit, ArCore plugins

[Nolkeg]

Could you please tell me, is it possible to create smt like this:

video_2021-05-11_20-48-30.mp4 First, you can move the circle on your table and then you can link it to a special point. After this you can open a model and it'll be downloaded and model will appear in the circle. The circle should be as a GameObject. If it's possible, could you tell me how to make it?

If you only need that functionality. There is no reason to use this plugin. Just go with ARFoundation. It's official package maintain & dev by unity.

[emreboyraz0]

I tried to erase everything except instant motion tracking, but application started to crash. I revert everything and after several experiments I find out that when we start Instant motion tracking as a first graph - application crashes but when I start another graph first and then I switch to Instant motion tracking- it works fine. Here are my crash logs: https://del.dog/ofignestaf.txt

BTW, I still don't know how to fix rotated camera

For Portrait mode give 90 to MainCamera's z index of rotation.

[homuler]

Fixed by #296

[Aashutosh-Dabhade]

Please see WebCamTexture#videoRotationAngle (though it may be a bug of sample apps)

Yes, it fixed camera rotation, but robot in instant motion tracking is still rotated, I don't know how to fix it

Hi! @Almazikman could you please explain how did you fix this issue?

[ROBYER1]

For the rotation, I was working on Mediapipe 0.8.4 at this commit for project stability reasons and this was how I approached the rotation issue on IOS and Android builds (Camera rotation was fine in editor). This solution may not apply to the newer version of the demo scenes.

1. Use webcamTexture.VideoRotationAngle to rotate the screen transform correctly on device.
2. In our case we were using the pose graph so when spawning the pose points, change the spawning from using transform.position to transform.localposition and spawn them with an offset transform parent that is parented to the camera gameobject in the scene.
3. Now you have the camera rotation correct, and the spawned pose points attached to an offset transform that you can rotate in code however you need. Once that is sorted, using transform.position to get the x,y,z coordinates of those pose points is also correct in its rotation in the scene view.