search

NumesSanguis / FACSvatar

An Open Source Modular Framework From Face to FACS Based Avatar Animation (Unity3D / Blender)
GNU Lesser General Public License v3.0
430 stars 97 forks source link

Feature Request: Reallusion iClone and Character Creator 3 support #18

Closed delebash closed 4 years ago

delebash commented 4 years ago

I am still trying to wrap my head around these types of programs but it would be great to have the ability to use any Character Creator 3 character and create facial animations that will work with that character in iClone.

https://www.reallusion.com/character-creator/ like MB-Lab

https://www.reallusion.com/iclone/ animation product that uses Character Creator 3 characters

I am not a really a programmer so setting up these types of projects but it would be nice to be able to use this with your own Characters or Characters from Daz,CC3,Poser. A one click import into which ever app you want to use. I can't help with the project from a programmers point of view but I would be willing to donate a few dollars for continued improvements. Ultimately I would like to be able to use FACSvatar, Openpose Openface or combination to easily create full body mocap from a video or webcam that can be used with any program or character. As a solo creator and newbie to 3D I can't afford a thousand dollars or more to get this functionality. So if I can help in any way to continue this project and hopefully expand to full body mocap please let me know!!

Thanks for the awesome work!

NumesSanguis commented 4 years ago

FACSvatar is a flexible way of transferring, and modifying data, from 1 program to another. So FACSvatar currently transfers the data from OpenFace to Blender or Unity3D. It works with FACShuman characters, because these characters have Blend Shapes / Shape Keys according to the Facial Action Coding System (FACS) specifications.

MB-Lab characters are supported because I wrote a conversion module that changes the output data of OpenFace (FACS-formatted) to data matching the Blend Shapes in characters created with MB-Lab.

If you want to support other characters, you have to do 2 things:

  1. Get the character imported in Blender (preferably), or Unity3D
  2. One of the following: a. 3D Modeller approach: Create a Face Rig with Blend Shapes that follow the FACS specifications. b. Write a module (like the process_facstoblend module) that converts FACS data to the blend shape data for facial configurations available in the character you imported.

I'm not a 3D modeller, but a programmer, so you would need to do step 2a yourself (or whomever is willing to help you). 2b is not difficult either in terms of coding. You'll only need to find the right mapping between FACS and the blend shapes available in that character. So that would be looking at the individual FACS videos and try to find which blend shape matches which Action Unit (AU) the best.

If you want to work with a different program than Blender / Unity3D, you would have to write code that accepts data from FACSvatar (support ZeroMQ) and transfer that received data to the character (which is program specific). This is likely going to take significant time to get it to work if you're not familiar with programming for that specific add-on. You would need to find your own developer for this if you can't code this yourself, because I'm not familiar with the programs you linked.

The reason why I created support for Unity3D was, was because for my research project (which has finished now) I needed a real-time game engine. Blender at that time didn't have the real-time EEVEE engine yet. The reason Blender is supported is because it's Open Source and in Python, which is the programming language I'm best at.

delebash commented 4 years ago

Thank you for the explanation and the help!