DanielSWolf / rhubarb-lip-sync

Rhubarb Lip Sync is a command-line tool that automatically creates 2D mouth animation from voice recordings. You can use it for characters in computer games, in animated cartoons, or in any other project that requires animating mouths based on existing recordings.
Other
1.72k stars 208 forks source link

Output format for Moho #69

Closed NorbertLinda closed 4 years ago

NorbertLinda commented 5 years ago

Would it be possible to implement an output-format for MOHO? This program (former animate studio) is very often used an some other programs use the same format. It is similar to the stanbdartd format but it uses frames instead of seconds. The first line of a Moho Switchdatafile (*.dat) must have the content "MohoSwitch1"

best regards Norbert Linda

DanielSWolf commented 5 years ago

I think writing a Moho exporter shouldn't be hard. The main problem for me is that I don't own Moho. So I wouldn't know whether the export works. Here are some points I'm not sure about:

If I had to convert the identifiers, this is how I'd probably do it:

Rhubarb identifier Moho identifier
A
MBP
B
etc
C
E
D
AI
E
U
F
WQ
G
FV
H
L
X
rest
n/a O
NorbertLinda commented 5 years ago

Dear Daniel,

Thank you for responding so quickly. Getting a 30 day full working demo for Moho 12 would be easy. This should be OK for testing. You can get it here: https://my.smithmicro.com/moho-pro-12-trial.html#trial I'm no Moho Professional, so i can't talk for all Moho-users, but these are my suggestions:

-There is no fixed frame rate. Users can set any frame rate in the project settings. I would suggest 24 as a default and any other value as a parameter.

-I'm not sure about the answer to the second question. As far as i know, the only content of a swith file i the time stamp in frames per seconds in the first row and the name of the correspondig image in the second row (and MooSwitch1 in the header). So Moho uses this for the selected frame rate in the project.

-Yes, Moho users often work with the papagayo-output and as far as i know, papagayo was initial written by someone from anime studio. The integrated Moho lip syncing features follow standards developed by Preston Blair too. But it is possible to use any files for the different mouthtypes, so it should be possible without any problems to use industy-standard too. The only thing to do is to rename the used grafics as expected. But in case of usabillity and acceptance it would be a good idea, having the possibilyty for a conversion e.g. to Preston Blair. The reason for this is the fact, that the users has build and named the files for the mouths for that, as you assumed. Your suggestion for a conversion is fine, a good idea.

DanielSWolf commented 5 years ago

Sorry for the late response. I think I will implement Moho support, but it might be a few weeks before I find the time.

NorbertLinda commented 5 years ago

Dear Daniel,

these are good news! I think a ot of users will be very happy about it.

best regards Norbert Linda

artisteacher commented 5 years ago

OpenToonz can use the .dat file for lip sync too. Since the last macOS Papagayo release is really old, it only partially functions. It would be great to have another option!

DanielSWolf commented 5 years ago

That's good to know! I'll try to make sure the output works with both programs.

I'm blocked right now because I can't get a functioning version of Moho. The demo is useless for my purposes -- it won't even allow me to import audio files. So I contacted Moho, asking if I could get a free version. That was ten days ago, and I haven't heard back from them.

If I'm to implement the Moho exporter, I need a functioning version of Moho so that I can test it, take screenshots for documentation, and maybe create a tutorial video. But I don't want to pay $70 for an application I won't use.

artisteacher commented 5 years ago

OpenToonz uses the exact same .dat file from Papagayo as Moho - so you could probably start the process with OpenToonz. It's free!

In OpenToonz, the default frame rate is 24fps but it can be changed. The frame rate is not included in the .dat file. If using a different frame rate than 24fps, you would probably have to set it in the lip-sync program and again in the OpenToonz scene for the timing to work.

Right now, OpenToonz uses Preston Blair identifiers for lip syncing.

NorbertLinda commented 5 years ago

As far as i know, the most open source programs are able to use the papagayo format. E.G OpenToonz is a candidate, synfig studio is another. I am sorry about the Moho-Trial, It have not known, that there is so much limitation. I can't understand, that you have not received a free version from Smith Micro, this is a bittle bit ignorant, if they really would not react.. By the way, a full version of Moho Pro costs about 400 Euro and 70 Euro for Moho Debut.

DanielSWolf commented 5 years ago

Great news: I contacted Smith Micro again on a different channel, and this time I received a very nice reply with a full Moho license. So I'll be able to test the new format with both applications after all! 😄

NorbertLinda commented 5 years ago

Hey...that's perfect. I was sure, that you will get a license. Everthing else would be crazy from Smith Micro in thei own interest. Good luck!

DanielSWolf commented 4 years ago

It took a while, but the exporter for Moho and OpenToonz is done. I'll release a new version soon.

artisteacher commented 4 years ago

I'm looking forward to trying this out. Are there any installation steps specific for macOS? I've unzipped the release folder and I'm using Terminal (on Mojave 10.14.5) but no matter what I try, all I get is:

-bash: rhubarb: command not found

Maybe I've missed a step?

DanielSWolf commented 4 years ago

Rhubarb is a command-line tool. You need to add the directory containing the rhubarb executable to your PATH variable. How exactly you do this depends on your version of OS X. Once you've done that, you should be able to call rhubarb.

artisteacher commented 4 years ago

Oh, that makes sense. Thanks! This works really well - and really speeds up lip-syncing in OpenToonz!