search

olexale / arkit_flutter_plugin

ARKit Flutter Plugin
MIT License
797 stars 225 forks source link

Request to access the scene class to map camera video to the 3D Face Geometry #91

Open Hemanshu-N-Solanki opened 4 years ago

Hemanshu-N-Solanki commented 4 years ago

Hi, I was wondering if you could allow access to the scene class as I would like to be able to use it to do this: Map Camera Video onto 3D Face Geometry as mentioned on the docs website: https://developer.apple.com/documentation/arkit/tracking_and_visualizing_faces I am able to complete everything else in this example except the line "material.diffuse.contents = sceneView.scene.background.contents" which requires access to the scene for which I cannot seem to find the given class in the package.

Thanks for your time, H

olexale commented 4 years ago

Hi @Hemanshu-N-Solanki,

It would be non-trivial to bridge the contents property of the scene's background as it might be pretty anything (a color, an image, a layer, a path, a SpriteKit scene, a texture, or a floating number). In case we make an assumption that it's a texture, the plugin needs to pack it on the iOS side, unpack on Flutter, set it to some property (for example the material of a face geometry), i.e. pack it on Flutter side, and unpack on iOS. I have concerns it will work fluently with 60 FPS on old iOS devices.

In any case, it is in the API, hence we need to make it work in the plugin as well, but it requires more investigations.

Meanwhile, just curious, are you just working through the sample, or you have a real-world scenario with this functionality?

Kind regards, Oleksandr

Hemanshu-N-Solanki commented 4 years ago

Hi Oleksandr, For now I'm working through the sample just to test if this was available on Flutter. However, I may require this functionality to have snapchat like functionality where face features can be manipulated in real time. Kind regards, Hemanshu

jjfster commented 1 year ago

I could really use access to the scene class too if possible. I'm used to doing this in swift:

`self.sceneView?.scene.background.contents =

                [
                    "1.png",
                    ...

            ]`

This works pretty well and is fast. But maybe there is another way to stitch a sky view in this plugin (or would I have to use the panorama sample type with sphere diffuse materials)?