YoungSeng / FreeTalker

Freetalker: Controllable Speech and Text-Driven Gesture Generation Based on Diffusion Models for Enhanced Speaker Naturalness (ICASSP 2024)
https://youngseng.github.io/FreeTalker/
62 stars 9 forks source link

FileNotFoundError #1

Closed jiaqiAA closed 9 months ago

jiaqiAA commented 9 months ago

Thanks for your great work!

No such file or directory: './data/prcocessed_data/v3_mean.npy' No such file or directory: './data/prcocessed_data/v3_std.npy'

YoungSeng commented 9 months ago

Thanks for the issue, I noticed some overlap in naming the folder data and dataset folders, however I just uploaded the ./data/prcocessed_data/v3_mean and v3_std, please check it out.

jiaqiAA commented 9 months ago

Thanks for your reply. when I run python -m sample.double_take --save_dir '' --guidacnce_param 1 --model_name model001000000 --BEAT_wav_feat ./datasets/BEAT/my_wav_feat/ --HUMANML3D_text_feat ./datasets/SMPLX/HumanML3D/v3_HUMANML3D_txt_feat/ --clip_model_path ./data/clip --vis_mode customized_controls, I meet an error:

ax.lines = []
AttributeError: can't set attribute

and I found sample_audio_feat, sample_text_feat, text, audio_tag = None, None, None, None https://github.com/YoungSeng/FreeTalker/blob/master/sample/double_take.py#L561 I'm a little confused about what this demo is for?

YoungSeng commented 9 months ago

Regarding the first question, it seems that there is an error in the visualization, can you provide more details about the error report? For example the full ERROR message; also for the second question, this is not a problem, as it is using processed speech features so it is all None here and does not affect the program operation. See

https://github.com/YoungSeng/FreeTalker/blob/77a4beb02f74524634ac4dd86471a02b6b7071cc/sample/double_take.py#L278

and

https://github.com/YoungSeng/FreeTalker/blob/77a4beb02f74524634ac4dd86471a02b6b7071cc/sample/double_take.py#L191

jiaqiAA commented 9 months ago

ERROR message

./save/my_v3_0/model001000000
Creating model and diffusion...
n_joints: 659, latent_dim: 256
TRANS_ENC init
model version v3_0
Loading checkpoints from [./save/my_v3_0/model001000000.pt]...
ViT-B/32 ./data/clip
guidacnce_param:  1.0
rec_ric_data shape:  torch.Size([630, 55, 3])
Traceback (most recent call last):
  File "anaconda3/envs/freetalker/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "anaconda3/envs/freetalker/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/FreeTalker/sample/double_take.py", line 579, in <module>
    process(sample_audio_feat, sample_text_feat, segment_length, save_path, model_name, n_seed=n_seed, version=version, text=text)
  File "/FreeTalker/sample/double_take.py", line 472, in process
    plot_3d_motion_1(os.path.join(save_path, "positions_vis1_{}_{}.mp4".format(guidacnce_param, args.vis_mode)), kinematic_chain, np.array(rec_ric_data), captions,
  File "/FreeTalker/process/plot_script.py", line 274, in plot_3d_motion
    explicit_plot_3d_motion(save_path, kinematic_tree, joints, title, dataset, figsize=figsize, fps=fps, radius=radius,
  File "/FreeTalker/process/plot_script.py", line 442, in explicit_plot_3d_motion
    ani.save(save_path, fps=fps, dpi=200)
  File "/anaconda3/envs/freetalker/lib/python3.8/site-packages/matplotlib/animation.py", line 1090, in save
    anim._init_draw()  # Clear the initial frame
  File "/anaconda3/envs/freetalker/lib/python3.8/site-packages/matplotlib/animation.py", line 1748, in _init_draw
    self._draw_frame(frame_data)
  File "/anaconda3/envs/freetalker/lib/python3.8/site-packages/matplotlib/animation.py", line 1767, in _draw_frame
    self._drawn_artists = self._func(framedata, *self._args)
  File "/FreeTalker/process/plot_script.py", line 385, in update
    ax.lines = []
AttributeError: can't set attribute
YoungSeng commented 9 months ago

Well I see, seemingly cause by ani.save(save_path, fps=fps, dpi=200). My matplotlib version is 3.1.3, can you try pip install matplotlib==3.1.3 and try again?

jiaqiAA commented 9 months ago

Thanks very much for your reply, it worked!

YoungSeng commented 9 months ago

Great, I also finished the update in readme.