search

Matamata-Animator / Matamata

Automatically create lip-synced animations
https://youtu.be/U4W1bv_cai0
72 stars 13 forks source link

Viseme? #56

Closed nitinthewiz closed 2 years ago

nitinthewiz commented 2 years ago

Love the work you’re doing. Just two questions -

  1. Under defaults you’ve called the json file phonemes.json. Shouldn’t it be visemes.json? That’s the actual name for the “images to represent sounds” sorta thing.

  2. Also, I just saw your intro video from 2021 talking about auto-lip-sync and you said that the mouths you’ve made are based on the Domic? Dominick? Set? Could you tell me what it is actually called? That part wasn’t clear in the video and I’d love to read up on that original set please.

effdotsh commented 2 years ago

Thanks for your interest in the project!

  1. Take this next part with a grain of salt, as I have never seen the word "Viseme" before today. As I understand it, a viseme is the name for the mouth shape and a phoneme is the name for the sound created. phonemes.json correlates the maps the phonemes to the visemes, so I think either name would work. Cool to remember just how surprisingly extensive this field is though!
  2. This isn't a paper or anything, this is just a reference to a video by a YouTuber, "Domics". He made a tutorial on how to hand draw lip-synced mouths, and I was saying that the mouths I drew took heavy inspiration from his set of drawings,
nitinthewiz commented 2 years ago

Thanks for the explanation! You are right, I guess. And frankly, it doesn’t matter. I just wanted to say hi 😊

and thanks for the link of the YouTuber. I was driving myself crazy googling for “phonemes domics set” and “domics viseme set”. Crazy!

Matamata is awesome! I’ll play with it a bit and see if I can help in any way.

Currently, I’m planning on using Wav2Lip for what I’m doing. But I really wanted to use visemes for animation. Your video introducing it (and how much work you’ve done on Matamata) might just help push me in this direction!

effdotsh commented 2 years ago

Yeah of course! If you ever wanna discuss this project or anything else really you can always join my discord server or just message me directly AI Spawn#7580.