search

Hujiazeng / Vach

Real time streaming talking head
341 stars 48 forks source link

请问为何没有声音，没去TTS？ #8

Closed wingjoezhou closed 1 month ago

wingjoezhou commented 1 month ago

(Vach) wing@DESKTOP-GF5P71Q:/data/Vach$ python app.py Block Mode: True Namespace(real_fps=15, mike=False, tts='edgetts', link_name='ErNerf', model_name='obama', base_dir='/data/Vach', block_mode=True, asr_model='cpierse/wav2vec2-large-xlsr-53-esperanto', preload=2, pose='/data/Vach/talkers/er_nerf/data/obama/transforms_train.json', au='/data/Vach/talkers/er_nerf/data/obama/au.csv', ckpt='/data/Vach/talkers/er_nerf/data/obama/ngp_kf.pth', template='/data/Vach/talkers/er_nerf/data/obama/template.npy', W=450, H=450, full_body=False, full_body_imgs='', torso_imgs='', bg_img='white', crop_x=0, crop_y=0, data_range=[0, 300], num_rays=65536, fbg=False, dt_gamma=0.00390625, O=False, amb_aud_loss=1, amb_dim=2, amb_eye_loss=1, asr=True, asr_save_feats=False, radius=3.35, seed=0, gui=False, fovy=21.24, lambda_amb=0.0001, max_ray_batch=4096, max_spp=1, max_steps=16, part=False, part2=False, test=True, update_extra_interval=16, upsample_steps=0, warmup_step=10000, fp16=True, color_space='srgb', fps=50, finetune_lips=False, asr_wav='', smooth_path=True, cuda_ray=True, asr_play=False, scale=4, torso_shrink=0.8, l=10, m=8, r=10, fix_eye=-1, aud='', offset=[0, 0, 0], init_lips=False, patch_size=1, exp_eye=True, smooth_eye=True, bound=1, min_near=0.05, density_thresh=10, density_thresh_torso=0.01, test_train=False, smooth_lips=True, smooth_path_window=7, ind_dim=4, ind_num=10000, ind_dim_torso=8, train_camera=False, emb=False, att=2, torso=True) Setting up [LPIPS] perceptual loss: trunk [alex], v[0.1], spatial [off] /data/miniconda3/envs/Vach/lib/python3.10/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and will be removed in 0.15, please use 'weights' instead. warnings.warn( /data/miniconda3/envs/Vach/lib/python3.10/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or None for 'weights' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passing weights=AlexNet_Weights.IMAGENET1K_V1. You can also use weights=AlexNet_Weights.DEFAULT to get the most up-to-date weights. warnings.warn(msg) Loading model from: /data/miniconda3/envs/Vach/lib/python3.10/site-packages/lpips/weights/v0.1/alex.pth [INFO] Trainer: ngp | 2024-05-25_02-55-47 | cuda | fp16 | None [INFO] #parameters: 1789121 [INFO] Loading /data/Vach/talkers/er_nerf/data/obama/ngp_kf.pth ... [INFO] loaded model. [INFO] load at epoch 28, global step 203616 [WARN] Failed to load optimizer. [INFO] loaded scheduler. [INFO] loaded scaler. [INFO] load 300 frames. Loading <class 'type'> data: 100%|█████████████████████████████████████| 300/300 [00:00<00:00, 88612.06it/s][INFO] eye_area: 0.0 - 0.9300000071525574 [INFO] loading ASR model cpierse/wav2vec2-large-xlsr-53-esperanto... /data/miniconda3/envs/Vach/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: resume_download is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use force_download=True. warnings.warn( /data/miniconda3/envs/Vach/lib/python3.10/site-packages/transformers/configuration_utils.py:364: UserWarning: Passing gradient_checkpointing to a config initialization is deprecated and will be removed in v5 Transformers. Using model.gradient_checkpointing_enable() instead, or if you are using the Trainer API, pass gradient_checkpointing=True in your TrainingArguments. warnings.warn( [INFO] warm up ASR live model, expected latency = 0.560000s [INFO] warm-up done, actual latency = 2.625267s load dataset use time 0.3011329174041748 start websocket server Traceback (most recent call last): File "/data/miniconda3/envs/Vach/lib/python3.10/site-packages/gevent/pywsgi.py", line 1107, in handle_one_response self.run_application() File "/data/miniconda3/envs/Vach/lib/python3.10/site-packages/geventwebsocket/handler.py", line 75, in run_application self.run_websocket() File "/data/miniconda3/envs/Vach/lib/python3.10/site-packages/geventwebsocket/handler.py", line 52, in run_websocket list(self.application(self.environ, lambda s, h, e=None: [])) File "/data/miniconda3/envs/Vach/lib/python3.10/site-packages/flask/app.py", line 1498, in call return self.wsgi_app(environ, start_response) File "/data/miniconda3/envs/Vach/lib/python3.10/site-packages/flask_sockets.py", line 40, in call handler, values = adapter.match() File "/data/miniconda3/envs/Vach/lib/python3.10/site-packages/werkzeug/routing/map.py", line 627, in match raise WebsocketMismatch() from None werkzeug.routing.exceptions.WebsocketMismatch: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand. 2024-05-24T18:56:14Z {'REMOTE_ADDR': '172.30.112.1', 'REMOTE_PORT': '60683', 'HTTP_HOST': '172.30.117.251:30003', (hidden keys: 30)} failed with WebsocketMismatch

Connection state is connecting Connection state is connected stop ! stop ! Connection state is closed Traceback (most recent call last): File "/data/miniconda3/envs/Vach/lib/python3.10/site-packages/gevent/pywsgi.py", line 1107, in handle_one_response self.run_application() File "/data/miniconda3/envs/Vach/lib/python3.10/site-packages/geventwebsocket/handler.py", line 75, in run_application self.run_websocket() File "/data/miniconda3/envs/Vach/lib/python3.10/site-packages/geventwebsocket/handler.py", line 52, in run_websocket list(self.application(self.environ, lambda s, h, e=None: [])) File "/data/miniconda3/envs/Vach/lib/python3.10/site-packages/flask/app.py", line 1498, in call return self.wsgi_app(environ, start_response) File "/data/miniconda3/envs/Vach/lib/python3.10/site-packages/flask_sockets.py", line 40, in call handler, values = adapter.match() File "/data/miniconda3/envs/Vach/lib/python3.10/site-packages/werkzeug/routing/map.py", line 627, in match raise WebsocketMismatch() from None werkzeug.routing.exceptions.WebsocketMismatch: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand. 2024-05-24T18:56:30Z {'REMOTE_ADDR': '172.30.112.1', 'REMOTE_PORT': '60709', 'HTTP_HOST': '172.30.117.251:30003', (hidden keys: 30)} failed with WebsocketMismatch

Connection state is connecting Connection state is connected

没有声音，小马口也没动。 我是用 WSL 2 的 ubuntu 22.04

Hujiazeng commented 1 month ago

降低flask 版本

wingjoezhou commented 1 month ago

降到了 1.1.4 就可以了， makeupsafe==2.0.1