Open Lcococi opened 5 months ago
same problem... spectre and EmoSwinModule. maybe probably related to this. https://github.com/filby89/spectre
https://github.com/radekd91/inferno/issues/7
last link not work
I successfully run “TalkingHead/demos/demo_eval_talking_head_on_audio.py" with EMOTE_v2. But the result was incorrect. Obvious Shaking! What's the problem?
https://github.com/radekd91/inferno/assets/138568445/21c73f0f-372c-41b8-8d83-540057b0857c
mark
The results are messy bad... Could you @radekd91 please check this? thx so much.
https://github.com/radekd91/inferno/assets/23353228/f6ea9407-7483-45fb-b5ea-dba21615fd5e
This looks like something is wrong with your mesh export. As if you were saving the vertices shuffled. Not quite sure why.
How did you arrive at this result exactly?
On Tue, Apr 30, 2024, 22:43 Liyun @.***> wrote:
The results are messy bad... Could you @radekd91 https://github.com/radekd91 please check this? thx so much.
https://github.com/radekd91/inferno/assets/23353228/f6ea9407-7483-45fb-b5ea-dba21615fd5e
— Reply to this email directly, view it on GitHub https://github.com/radekd91/inferno/issues/9#issuecomment-2088012287, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA7MKHPXHJ5J6YQR4TLT2CTZAB6IRAVCNFSM6AAAAABCWKKOUKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBYGAYTEMRYG4 . You are receiving this because you were mentioned.Message ID: @.***>
I successfully run “TalkingHead/demos/demo_eval_talking_head_on_audio.py" with EMOTE_v2. But the result was incorrect. Obvious Shaking! What's the problem? pyrender_with_audio.mp4
@yujia-mt this is definitely wrong. How did you call the demo? It should look like this:
https://github.com/radekd91/inferno/assets/4113693/ff6dad29-002a-43af-b1d6-8ab470c8a27b
Can you share the std output? It should print somewhere what was loaded (which checkpoints). This looks like an unconverged model.
I encountered the same error. It seems to occur during the final rendering and composition of the video. Importing the obj files from "M005_Happy_0/meshes" into Blender works fine.
The results are messy bad... Could you @radekd91 please check this? thx so much.
pyrender_with_audio.mp4
@zhangliyun9120 @yinfan-24 I encountered the same error and fix it. It seems like the predicted vertices need to be assigned to the deep copy object of head template.obj, then converting this new psbody.mesh.Mesh object to trimesh.Trimesh object and rendering it.
pred_vertices = predicted_vertices[t].detach().cpu().view(-1,3).numpy() mesh = copy.deepcopy(template_obj_ps) mesh.v = pred_vertices trimesh_mesh = trimesh.Trimesh(vertices=mesh.v, faces=template.faces) pred_image = renderer.render(trimesh_mesh)
The parameter type of the render function should be modified.
Could not import SPECTRE. Make sure you pull the repository with submodules to enable SPECTRE. Traceback (most recent call last): File "/mnt/workspace/inferno/inferno/models/temporal/external/SpectrePreprocessor.py", line 16, in
from spectre.src.spectre import SPECTRE
ModuleNotFoundError: No module named 'spectre.src'
Could not import EmoSwinModule. SWIN models will not be available. Make sure you pull the repository with submodules to enable SWIN. Could not import EmoSwinModule. SWIN models will not be available. Make sure you pull the repository with submodules to enable SWIN. SWIN not found, will not be able to use SWIN models Looking for checkpoint in '/mnt/workspace/inferno/assets/TalkingHead/models/EMOTE_v2/checkpoints' Found 1 checkpoints