Whether to support sound streaming input audio2face to drive lip shape？

[CasonTsai]

thank you for your great work，May I ask if this project supports sound streaming into audio2face to drive lip shapes, rather than in the form of voice files?

[w4hns1nn]

Hey,

I haven't implemented that yet, however I think it's not that complicated, the headless server of Nvidia supports it. Feel free to contribute, I would appreciate!

Best regards

[singelhero]

感谢无私奉献，能否进一步实现实时流式驱动？

[w4hns1nn]

Confirmed. Will work on it. The project will be improved and continued.

[w4hns1nn]

Started working on it. In a second step I will also move away from plain requests and build on fastSDK and media-toolkit. Second step will take longer though. I'll release an upgraded module first - can't tell you yet how long it takes, but I expect to finish in a few weeks.

[w4hns1nn]

@singelhero @CasonTsai can you give me a little bit more information where your input stream comes from? Do you expect a live microphone audio input? What are your aims / projects you're working on about? How would this feature look like in your opinion?

For the beginning I would implement chunking an audio file and feeding it to the stream.

[w4hns1nn]

Finished streaming implementation. @singelhero, @CasonTsai please check feature especially interesting wiht livelink. Please share result. Raise new issue if something is missing or misbehaving. Next steps: fastSDK

[CasonTsai]

@w4hns1nn yes,

@singelhero @CasonTsai can you give me a little bit more information where your input stream comes from? Do you expect a live microphone audio input? What are your aims / projects you're working on about? How would this feature look like in your opinion?

For the beginning I would implement chunking an audio file and feeding it to the stream.

@singelhero @CasonTsai can you give me a little bit more information where your input stream comes from? Do you expect a live microphone audio input? What are your aims / projects you're working on about? How would this feature look like in your opinion?

For the beginning I would implement chunking an audio file and feeding it to the stream.

yeah ,i want to build a metahuman in unreal.the input stream from microphone audio input or audio data in memory .i found a solution in audio2face official doc,it use grpc in that document. excuse me,I want to know if there is a way to use audio2face without opening omniverse?

[w4hns1nn]

@CasonTsai I've added the streaming feature which uses gRPC. Check the updated readme for it. About the second question:

• please check if the headless server also uses omniverse.. py_audio2face runs with the headles..
• I plan something similar, but will be end of the year until i can publish something about

[CasonTsai]

@CasonTsai I've added the streaming feature which uses gRPC. Check the updated readme for it. About the second question:

• please check if the headless server also uses omniverse.. py_audio2face runs with the headles..
• I plan something similar, but will be end of the year until i can publish something about

thanks