search

virtual-puppet-project / vpuppr

VTuber application made with Godot 4
Mozilla Public License 2.0
726 stars 62 forks source link

Eyebrow Tracking #110

Open ja-cop opened 2 years ago

ja-cop commented 2 years ago

I see the eyebrow tracking from OSF is referenced in the code, but I guess it's not fully implemented? Apparently it's not a VRM standard, but VSeeFace implements this for VRM models with two custom blendshape clips, "Brows up" and "Brows down": https://www.vseeface.icu/#special-blendshapes

Combined with #105 I'm sure it could bring a lot of life to some models! :slightly_smiling_face:

you-win commented 2 years ago

I'm a little hesitant to implement features that aren't defined by a specification, otherwise there will be no end to random feature requests. Currently the all/most of the VRM specification is implemented. In theory, Perfect Sync models generated by the HANA tool are almost supported (a few tweaks are needed) which has 51(?) facial blendshapes

ItsRogueRen commented 2 years ago

I would also be interested in the eyebrow tracking since OSF does provide it, even if it's not VRM standard.

ja-cop commented 2 years ago

I'm a little hesitant to implement features that aren't defined by a specification, otherwise there will be no end to random feature requests. Currently the all/most of the VRM specification is implemented. In theory, Perfect Sync models generated by the HANA tool are almost supported (a few tweaks are needed) which has 51(?) facial blendshapes

I completely understand this, but to get Perfect Sync, I need to send tracking data from an Apple device, right? Or does OpenSeeFace provide enough tracking data for Perfect Sync? I'm not very familiar with the options in this space.

As an aside, as far as I understand, I don't think I can use Hana Tool for my models as they're not based on VRoid Studio. That said, I would be completely willing to set up the Perfect Sync blendshapes myself if OpenSeeFace can handle it.

I haven't actually tried the VSeeFace eyebrow feature, but I got the impression that it actually works with OpenSeeFace without additional tracking data, which is presumably what justifies the feature.

you-win commented 2 years ago

@ItsRogueRen @ja-cop I didn't dawn on me that the feature request is because OpenSeeFace already supports the feature. I added this to the list of things that still need to be done.

you-win commented 1 year ago

Kind of added in 8b3d947, this does require the standalone iOS app "iFacialMocap" to work + a vrm model converted to a Perfect Sync model. Will continue looking at implementing this for OpenSeeFace