search

SceneView / sceneview-android

SceneView is a 3D and AR Android Composable and View with Google Filament and ARCore. This is a Sceneform replacement in Kotlin
Apache License 2.0
844 stars 161 forks source link

Ar faces #299

Closed TooLazyy closed 10 months ago

TooLazyy commented 1 year ago

Hi Any sample of how to implement ar faces feature? I.e. load a model, track face, place model on the face.

grassydragon commented 1 year ago

Hi! As far as I remember, neither we've had time to add a sample for Augmented Faces and address any related issues in the library nor anyone else has succeeded in doing that.

TooLazyy commented 1 year ago

Yo. Thanks for the answer. Any plans to add AR faces support? Or maybe some implementation helps so I do it on my own?

Blizl commented 1 year ago

I found one here: https://github.com/SceneView/sceneform-android/tree/master/samples/augmented-faces, but this is from the old sceneform repo.

EDIT: Looks like I misunderstood the codebase. Maybe adding a AugmentedFaces will be helpful like in the old sceneform repo. Will look into seeing if I can get it done. Still fairly new to the codebase.

TooLazyy commented 1 year ago

Yeah, I was able to run the old sample, but this lib seem to be not supporting it yet. Would be very nice to add

bankbiz commented 1 year ago

@grassydragon Can you guide me a little bit to finish the implementation and open PR for this feature? Because I saw the AugmentedFaceMode config and onAugmentedFaceUpdate property. A guide from you may help me understand the library code base faster.

Thanks.

grassydragon commented 1 year ago

Hi @bankbiz! Have you already started to write your implementation? There have been previous attempts by other contributors but they haven't been able to work around some issues. I think the ideal way would be to have a tech talk on our Discord server, however, I need to recollect how the Augmented Faces work and what were the encountered issues :sweat_smile:

hshapley commented 1 year ago

I have tried to get Augmented Faces working myself. If you copy the AugmentedFaceNode code from Sceneform and make some small fixes, you can get a 3D model to track with the face. The main part that doesn't work is the face occlusion, because Sceneform used an old method of building the mesh that doesn't seem to be supported any more. Now that Geometry has been introduced in Sceneview, it should be possible to rebuild that part, although it'll be hard. I was planning to add AugmentedFaces support when I get some free time, but I don't know when that will be. I could at least add a stripped-down AugmentedFaceNode class that only supports the tracking. Oh, and putting textures on faces also doesn't work, but I never used that so I don't know how to bring it back either.

hshapley commented 1 year ago

Post-0.10.0, models are really tiny and/or far away when using face tracking / the front-facing camera, whereas that wasn't a problem before. That'd need to be fixed for augmented faces to be viable.

TooLazyy commented 1 year ago

Hi. Any updates on this? Maybe I can help you?

github-actions[bot] commented 10 months ago

Hey there, it looks like there has been no activity on this issue recently. Has the issue been fixed, or does it still require the community's attention? This issue may be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 10 months ago

Closing this issue after a prolonged period of inactivity. If this issue is still present in the latest release, please feel free to create a new issue with up-to-date information.