bornfree / talking_avatar

A nice 3D avatar that can speak input text with facial expressions
MIT License
288 stars 88 forks source link

cuustom character #1

Open toozande opened 1 year ago

toozande commented 1 year ago

hello this is awesome project how can I create my own character? I need Men , Women and child character regards

bornfree commented 1 year ago

Unfortunately the process to create a character with the correct blendshapes is quite cumbersome. I'll record a video and share it. Maybe that can help.

microscopes2468 commented 1 year ago

Unfortunately the process to create a character with the correct blendshapes is quite cumbersome. I'll record a video and share it. Maybe that can help.

I also want to know how to do custom characters.

longdang2709 commented 1 year ago

@bornfree I love your project too and would like to learn how to create a custom character and theme. Please share

premiumleads commented 1 year ago

Unfortunately the process to create a character with the correct blendshapes is quite cumbersome. I'll record a video and share it. Maybe that can help.

Yes please we need to put video avatars

jahnaviraj commented 1 year ago

Please share the video. Also did you make this character or you downloaded it from somewhere? @bornfree

lukevink-dentsu commented 1 year ago

Hey @bornfree can you share that video? Or perhaps you could share the blender file so we can understand how to? I am very curious how you rigged it etc. Cheers

ghost commented 1 year ago

Hi @bornfree how did you trained the model ? Is there any resource to follow ?

brianmabua commented 1 year ago

Hello, @bornfree. Can you point me to where I can information on how to customise the avatar?

guilhermelia commented 1 year ago

The website he used to create avatar is unavailable (3dify.me). Maybe never come back again. Sorry.

robegamesios commented 1 year ago

hi @bornfree can show us how to change the avatar and voice please. thanks

robegamesios commented 1 year ago

Hey thanks again for this. I used it as a starting point to integrate GPT, I can talk to it now. I still haven’t figured out how to change the avatar though, I tried modifying the glb in blender but it breaks when I export it again. Anyway here is the demo https://youtu.be/iv8bP2X0IQM

rezadadashian2 commented 1 year ago

Hi @bornfree. Any update?

Hey thanks again for this. I used it as a starting point to integrate GPT, I can talk to it now. I still haven’t figured out how to change the avatar though, I tried modifying the glb in blender but it breaks when I export it again. Anyway here is the demo https://youtu.be/iv8bP2X0IQM

Hi Rob, did you find a way to generate or create your own avatar (model)? We've been trying for days and it was not successful yet.

robegamesios commented 1 year ago

Hi @bornfree. Any update?

Hey thanks again for this. I used it as a starting point to integrate GPT, I can talk to it now. I still haven’t figured out how to change the avatar though, I tried modifying the glb in blender but it breaks when I export it again. Anyway here is the demo https://youtu.be/iv8bP2X0IQM

Hi Rob, did you find a way to generate or create your own avatar (model)? We've been trying for days and it was not successful yet.

hi, nope I’ve tried for a week but not easy. I’ve tried replacing the model using a rigged avatar but no luck. I also tried to reuse the existing model’s rig and apply to a different model. I even tried just modifying the existing avatar’s face in blender but it stopped working after. There are ‘cold targets’ in the avatar that are mapped to the face. I’m gonna try to recreate those targets on a different model and see if that works (mimicing how this existing model was made) probably next week when I free up. If you know blender then you can recreate it (I think). I’ll keep you posted if I get it to work.

rezadadashian2 commented 1 year ago

@robegamesios just exactly like us :))))) we went the same way. We have a 3D designer on our team but i had no luck interacting with her about this project so i'm starting to learn Blender myself. I'll let you know too if i find something worthy.

robegamesios commented 1 year ago

@robegamesios just exactly like us :))))) we went the same way. We have a 3D designer on our team but i had no luck interacting with her about this project so i'm starting to learn Blender myself. I'll let you know too if i find something worthy.

Fyi, I tried importing the model.glb and exporting it back out again using blender without even modifying anything and it’s totally broke the model. The head position looks up like in a horror film and I get errors when running it in threejs. I tried all the export glb options in blender and nothing worked for me. Have you tried this?

rezadadashian2 commented 1 year ago

@robegamesios We managed to make it work by creating a new avatar on our own. I've sent you an Email for the instructions.

robegamesios commented 1 year ago

@robegamesios We managed to make it work by creating a new avatar on our own. I've sent you an Email for the instructions.

Thanks! Did you have to change any of the default export to glb settings in blender?

rezadadashian2 commented 1 year ago

@robegamesios No, the default export settings are just fine.

jahnaviraj commented 1 year ago

@robegamesios We managed to make it work by creating a new avatar on our own. I've sent you an Email for the instructions.

Hi, Could you please share the intructions with me also? I would appreciate it :)

rezadadashian2 commented 1 year ago

@jahnaviraj For starter, create a simple model in Blender, then you can use a practical add-on called "Faceit" to make all the required 52 "Shape Keys" (these are the keys that has the same name with Azure API keys) for the face in less than an hour. Then export the model in .glb file with default export settings. Just remember to comment-out unnecessary codes which make the project break. for example you have to comment this code for now because most of the simple avatars doesn't have a LowerTeeth (LowerJaw is correct): // createAnimation(blendData, morphTargetDictionaryLowerTeeth, 'HG_TeethLower') if you have TeethLower (Lower Jaw) then you use Faceit to create Shape Keys for that too.

I do not recommend using the model in this project as it breaks how ever you export it.

robegamesios commented 1 year ago

@robegamesios We managed to make it work by creating a new avatar on our own. I've sent you an Email for the instructions.

Hi, Could you please share the intructions with me also? I would appreciate it :)

Also if you have an existing rigged model, you can try to rename the shapekeys to match the azure blendshape names. Easiest test is to rename the open mouth to jawOpen and export to glb binary and see if it works before renaming the rest. The cleanest way is to do as @rezadadashian2 mentioned above. I tried this on an existing model but since the file size is huge, it affects the initial loading time of the browser . I also got some unwanted side effects like the teeth went missing on my exported glb and I’m not sure why

toozande commented 1 year ago

@jahnaviraj For starter, create a simple model in Blender, then you can use a practical add-on called "Faceit" to make all the required 52 "Shape Keys" (these are the keys that has the same name with Azure API keys) for the face in less than an hour. Then export the model in .glb file with default export settings. Just remember to comment-out unnecessary codes which make the project break. for example you have to comment this code for now because most of the simple avatars doesn't have a LowerTeeth (LowerJaw is correct): // createAnimation(blendData, morphTargetDictionaryLowerTeeth, 'HG_TeethLower') if you have TeethLower (Lower Jaw) then you use Faceit to create Shape Keys for that too.

I do not recommend using the model in this project as it breaks how ever you export it.

good job but do you know if there is free API like Azure API key ?

rezadadashian2 commented 1 year ago

@jahnaviraj For starter, create a simple model in Blender, then you can use a practical add-on called "Faceit" to make all the required 52 "Shape Keys" (these are the keys that has the same name with Azure API keys) for the face in less than an hour. Then export the model in .glb file with default export settings. Just remember to comment-out unnecessary codes which make the project break. for example you have to comment this code for now because most of the simple avatars doesn't have a LowerTeeth (LowerJaw is correct): // createAnimation(blendData, morphTargetDictionaryLowerTeeth, 'HG_TeethLower') if you have TeethLower (Lower Jaw) then you use Faceit to create Shape Keys for that too. I do not recommend using the model in this project as it breaks how ever you export it.

good job but do you know if there is free API like Azure API key ?

@toozande Azure text-to-speech API is free. maybe you wanted to ask about open-source? No I don't know any.

VanshRana12 commented 1 year ago

@jahnaviraj For starter, create a simple model in Blender, then you can use a practical add-on called "Faceit" to make all the required 52 "Shape Keys" (these are the keys that has the same name with Azure API keys) for the face in less than an hour. Then export the model in .glb file with default export settings. Just remember to comment-out unnecessary codes which make the project break. for example you have to comment this code for now because most of the simple avatars doesn't have a LowerTeeth (LowerJaw is correct): // createAnimation(blendData, morphTargetDictionaryLowerTeeth, 'HG_TeethLower') if you have TeethLower (Lower Jaw) then you use Faceit to create Shape Keys for that too.

I do not recommend using the model in this project as it breaks how ever you export it.

Thanks a lot! Gonna try this.

subinoybiswas commented 1 year ago

@robegamesios We managed to make it work by creating a new avatar on our own. I've sent you an Email for the instructions.

Can you send it to me also?

Shivamkumar285 commented 12 months ago

@robegamesios We managed to make it work by creating a new avatar on our own. I've sent you an Email for the instructions.

Can you please send the instructions as well to me? Might be some repo that you have created?

rezadadashian2 commented 11 months ago

@Shivamkumar285 There's lots of things you have to consider while creating the avatar, therefore I won't be able to tell it all here and there's no public repository i currently would have for this.

But, the best clue i can you is to use a tool named "FaceIt" on Blender to assign each and every shape key the developer used in this repository to make the avatar talk (and 90% of all errors are related to shape keys and talking functionality)

MS-YUN commented 9 months ago

@robegamesios We managed to make it work by creating a new avatar on our own. I've sent you an Email for the instructions.

Hello. Can you send it to me as well? I didn't know the graphics at all, but I've tried various methods while studying for a month, but I'm constantly getting lost. I sincerely ask you to do that. Thank you.

Utkarshkapil commented 9 months ago

bro died after creating repo lmao

itsaizaz commented 6 months ago

We're excited to announce that we've recently enhanced our system with some cutting-edge new features! First, we've integrated an AI chatbot that's designed to provide you with faster, more efficient support and assistance. This chatbot is equipped with advanced natural language processing capabilities, ensuring a smooth and intuitive interaction experience. Additionally, we've revamped our model design, introducing a sleeker, more user-friendly interface that not only looks better but also enhances usability. These updates are part of our ongoing commitment to improving user experience and delivering value through innovative solutions. We can't wait for you to try them out!

ssreeram commented 5 months ago

@robegamesios We managed to make it work by creating a new avatar on our own. I've sent you an Email for the instructions.

Can you please share the instruction to shyam.sreeram@gmail.com please, i stuck at the same place

itsaizaz commented 5 months ago

@robegamesios We managed to make it work by creating a new avatar on our own. I've sent you an Email for the instructions.

Can you please share the instruction to shyam.sreeram@gmail.com please, i stuck at the same place

hey bro I can help you

ssreeram commented 5 months ago

@itsaizaz Please let me know how to do this? can u share the instructions on above shared email? please

ssreeram commented 5 months ago

@robegamesios We managed to make it work by creating a new avatar on our own. I've sent you an Email for the instructions.

Can you please share the instruction to shyam.sreeram@gmail.com please, i stuck at the same place

hey bro I can help you

Can u help me, stuck in generating the new character working with lip sync

itsaizaz commented 5 months ago

@itsaizaz Please let me know how to do this? can u share the instructions on above shared email? please

check your email

eyperrone commented 5 months ago

@itsaizaz please can you help me to do this? my email is ivanperrone1@gmail.com thanks!

TeemuSo commented 5 months ago

@itsaizaz Could you please send me some instructions on how to do it? I would appreciate a lot. teemus8281@gmail.com

raimondogiordano commented 5 months ago

@itsaizaz Please let me know how to do this? can u share the instructions on above shared email? please

check your email

@itsaizaz Can you please send also to me the instructions? my email is raimondogiordano4@gmail.com

sunilsingh2019 commented 1 month ago

Hey @itsaizaz, @robegamesios can you please share me instruction to create character ? sunilsingh2019@gmail.com

FraktalPL commented 1 month ago

Hey @itsaizaz, @robegamesios i need instructions to create new avatar too skakunypl@gmail.com If You can send me info how to integrate chat gpt with avatar? Greetings

anujsoni05 commented 3 weeks ago

Hey @itsaizaz Sir, @robegamesios Sir can you share instruction how to change the avatar or create character ? mail id :anujsoni93837@gmail.com

vnyash commented 1 week ago

hey @itsaizaz, can you share me instructions to change the avatar? mail : vnyash04@gmail.com