search

NumesSanguis / FACSvatar

An Open Source Modular Framework From Face to FACS Based Avatar Animation (Unity3D / Blender)
GNU Lesser General Public License v3.0
436 stars 99 forks source link

[FeatureRequest] Input: csv & fbx -> Output fbx_blendshape #13

Open GermanCreator opened 5 years ago

GermanCreator commented 5 years ago

Please create a cli where you input your csv and your face-fbx or body+face-fbx and it outputs a fbx with blendshapes applied, with rotating eyes and head movement data.

Alternatively do it with dae.

NumesSanguis commented 5 years ago

I definitely see the value of simplifying the process and allowing for scripted generation of .fbx / .dae files with facial animation included.

I think this is best implemented with the Blender add-on for FACSvatar, that is in development. Functions, like "choose .csv", would be added to the add-on. This add-on would then pass on the values to the FACSvatar modules outside Blender. These modules do their processing as usual and Blender waits for the resulting data to be streamed back into the avatar.

If this is all possible from the interface of the add-on, it means you can also call them from a script that calls Blender and passes on the variables you need. It would be the same CLI as you would normally use to do Blender tasks without interface.

Since this is a community project, it's not that I'm going to implement this all by myself. In the end it's up to the community members to add functionality for what they need, but I'm glad to help and discuss. I hope to add functionality that's useful for everyone, but in the end this is a side project for me and I've limited time to work on this. So expect that you, or some of your colleagues / someone you hire, to code, etc.

If you're still interested, we can discuss how to make this possible in more detail?

GermanCreator commented 5 years ago

Your project is shattered into pieces that are linked together. Even though it is working, it is hard to impossible to follow your approach. I can understand that you do this in your free time and therefore want to encourage collaboration. But to collaborate/contribute, your code structure needs to be more coherent. So either you restructure/refactor your project, or the cli is dependent on your willingness to create.

NumesSanguis commented 5 years ago

I'm aware my project is not the easiest to start with. The flexible nature makes it powerful, but harder to grasp. Also, it connects software I don't manage (e.g. OpenFace, ManuelBastioniLAB, FACSHuman). What part are you struggle with? I hope with docker-compose at least getting everything to run is quite straightforward. What part needs to be more coherent / clearer? How to develop new modules? How to have ZeroMQ sockets communicate with each other? The communication paradigm of the current modules? How to connect it with Blender (which I agree, is definitely a mess now)? Some part of the documentation?

GermanCreator commented 5 years ago

The cli I mentioned is also a good start for this. Have one python module or better, executable for windows, that has several inputs and outputs like the one mentioned. Include libraries for fbx import export rather than depending on bulky systems like openface, unity and blender. Then opensource this. THEN people will contribute. Because remember, it's not just your free time, it's also theirs. Your free time to develop, their free time to understand your code and contribute.

BTW: you have a .ch email. Are you from the german speaking part?

NumesSanguis commented 5 years ago

I'm sorry, if you're looking for that kind of tool, you're not the audience of this project. You want a ready made project, which is more likely what a commercial company would offer. If you want to redo the functionality of long standing projects like OpenFace, Unity3D and Blender, alone, it would take a full-time job for more than a year, and likely it would result in a project inferior in quality than all 3 of them, which would result in no one using it.

The whole design idea behind modules is that you can replace a part of it with some new better open source project that e.g. does face tracking. You only have to replace 1 module, without having to bother about code in the other modules. It is also code language independent, because not everything is made in Python. If you don't have a whole dedicated developer team, this is one of the fastest ways, because you just have to create a data interface (ZeroMQ) for a new project, instead of recreating the same functionality from scratch.

You don't have to understand the code in other modules, only how to start them. If you want to develop something new, you only need to know how to receive data from another module and how to pass it on. So for adding functionality to Blender, you only need to know how to send a path to a .csv to the input_facsfromcsv module and how to receive the resulting data back in Blender from process_facstoblend (which will be shown in the upcoming Blender add-on).

p.s. I'm not from Switzerland. ProtonMail can be made accounts can be made by everyone: https://protonmail.com/ . My German is quite limited.