Zz-ww / SadTalker-Video-Lip-Sync

本项目基于SadTalkers实现视频唇形合成的Wav2lip。通过以视频文件方式进行语音驱动生成唇形,设置面部区域可配置的增强方式进行合成唇形(人脸)区域画面增强,提高生成唇形的清晰度。使用DAIN 插帧的DL算法对生成视频进行补帧,补充帧间合成唇形的动作过渡,使合成的唇形更为流畅、真实以及自然。
1.8k stars 308 forks source link

使用use_DAIN参数时报错, error when take parameter --user_DAIN #48

Open shermanlsh opened 1 year ago

shermanlsh commented 1 year ago

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮ │ D:\soundmaker\SadTalker-Video-Lip-Sync\inference.py:124 in │ │ │ │ 121 │ │ args.device = "cuda" │ │ 122 │ else: │ │ 123 │ │ args.device = "cpu" │ │ ❱ 124 │ main(args) │ │ 125 │ │ │ │ D:\soundmaker\SadTalker-Video-Lip-Sync\inference.py:87 in main │ │ │ │ 84 │ │ predictor_dian = dain_predictor.DAINPredictor(args.dian_output, weight_path=args │ │ 85 │ │ │ │ │ │ │ │ │ │ │ │ │ time_step=args.time_step, │ │ 86 │ │ │ │ │ │ │ │ │ │ │ │ │ remove_duplicates=args.remove_dupl │ │ ❱ 87 │ │ frames_path, temp_video_path = predictor_dian.run(tmp_path) │ │ 88 │ │ paddle.disable_static() │ │ 89 │ │ save_path = return_path[:-4] + '_dain.mp4' │ │ 90 │ │ command = r'ffmpeg -y -i "%s" -i "%s" -vcodec copy "%s"' % (temp_video_path, new │ │ │ │ D:\soundmaker\SadTalker-Video-Lip-Sync/src/..\src\dain_model\dain_predictor.py:166 in run │ │ │ │ 163 │ │ │ │ │ 164 │ │ │ X = np.concatenate((X0, X1), axis=0) │ │ 165 │ │ │ │ │ ❱ 166 │ │ │ o = self.baseforward(X) │ │ 167 │ │ │ │ │ 168 │ │ │ y = o[0] │ │ 169 │ │ │ │ D:\soundmaker\SadTalker-Video-Lip-Sync/src/..\src\dain_model\base_predictor.py:44 in │ │ base_forward │ │ │ │ 41 │ │ │ else: │ │ 42 │ │ │ │ feed_dict[self.feed_names[0]] = inputs │ │ 43 │ │ │ │ │ ❱ 44 │ │ │ out = self.exe.run(self.program, │ │ 45 │ │ │ │ │ │ │ fetch_list=self.fetch_targets, │ │ 46 │ │ │ │ │ │ │ feed=feed_dict) │ │ 47 │ │ │ │ D:\sd-webui-aki-v4\py310\lib\site-packages\paddle\fluid\executor.py:1463 in run │ │ │ │ 1460 │ │ │ core.update_autotune_status() │ │ 1461 │ │ │ return res │ │ 1462 │ │ except Exception as e: │ │ ❱ 1463 │ │ │ six.reraise(*sys.exc_info()) │ │ 1464 │ │ │ 1465 │ def _run_impl(self, program, feed, fetch_list, feed_var_name, │ │ 1466 │ │ │ │ fetch_var_name, scope, return_numpy, use_program_cache, │ │ │ │ D:\sd-webui-aki-v4\py310\lib\site-packages\six.py:719 in reraise │ │ │ │ 716 │ │ │ │ value = tp() │ │ 717 │ │ │ if value.traceback is not tb: │ │ 718 │ │ │ │ raise value.with_traceback(tb) │ │ ❱ 719 │ │ │ raise value │ │ 720 │ │ finally: │ │ 721 │ │ │ value = None │ │ 722 │ │ │ tb = None │ │ │ │ D:\sd-webui-aki-v4\py310\lib\site-packages\paddle\fluid\executor.py:1450 in run │ │ │ │ 1447 │ │ │ self._log_force_set_program_cache(use_program_cache) │ │ 1448 │ │ │ │ 1449 │ │ try: │ │ ❱ 1450 │ │ │ res = self._run_impl(program=program, │ │ 1451 │ │ │ │ │ │ │ │ feed=feed, │ │ 1452 │ │ │ │ │ │ │ │ fetch_list=fetch_list, │ │ 1453 │ │ │ │ │ │ │ │ feed_var_name=feed_var_name, │ │ │ │ D:\sd-webui-aki-v4\py310\lib\site-packages\paddle\fluid\executor.py:1661 in _run_impl │ │ │ │ 1658 │ │ │ │ else: │ │ 1659 │ │ │ │ │ tensor._copy_from(cpu_tensor, self.place) │ │ 1660 │ │ │ │ │ ❱ 1661 │ │ │ return new_exe.run(scope, list(feed.keys()), fetch_list, │ │ 1662 │ │ │ │ │ │ │ return_numpy) │ │ 1663 │ │ │ │ 1664 │ │ compiled = isinstance(program, compiler.CompiledProgram) │ │ │ │ D:\sd-webui-aki-v4\py310\lib\site-packages\paddle\fluid\executor.py:631 in run │ │ │ │ 628 │ │ """ │ │ 629 │ │ fetch_list = self._check_fetch(fetch_list) │ │ 630 │ │ │ │ ❱ 631 │ │ tensors = self._new_exe.run(scope, feed_names, │ │ 632 │ │ │ │ │ │ │ │ │ fetch_list)._move_to_list() │ │ 633 │ │ if return_numpy: │ │ 634 │ │ │ return as_numpy(tensors, copy=True) │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ ValueError: In user code:

File "/root/miniconda3/lib/python3.7/site-packages/paddle/fluid/framework.py", line 2798, in append_op
attrs=kwargs.get("attrs", None))

File "/root/miniconda3/lib/python3.7/site-packages/paddle/fluid/layer_helper.py", line 43, in append_op
return self.main_program.current_block().append_op(*args, **kwargs)

File "/root/miniconda3/lib/python3.7/site-packages/paddle/fluid/layers/tensor.py", line 1412, in range
outputs={'Out': out})

File "/paddle/work/github/DAIN/DAIN-paddle-release/PWCNet/PWCNet.py", line 125, in warp
bb = fluid.layers.range(0, B, 1, 'float32')

File "/paddle/work/github/DAIN/DAIN-paddle-release/PWCNet/PWCNet.py", line 292, in forward
warp5 = self.warp(c25, up_flow6 * 0.625)

File "/root/miniconda3/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 600, in __call__
outputs = self.forward(*inputs, **kwargs)

File "/paddle/work/github/DAIN/DAIN-paddle-release/networks/DAIN_slowmotion.py", line 136, in forward_flownets
temp = model(input)

File "/paddle/work/github/DAIN/DAIN-paddle-release/networks/DAIN_slowmotion.py", line 72, in forward
self.forward_flownets(self.flownets, cur_offset_input, time_offsets=time_offsets),

File "/root/miniconda3/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 600, in __call__
outputs = self.forward(*inputs, **kwargs)

File "parse_weights.py", line 25, in convert_model
out = DAIN(image)

File "parse_weights.py", line 179, in <module>
convert_model(pkl_path, model_path)

InvalidArgumentError: The step should be greater than 0 while start < end.
  [Hint: Expected step > 0, but received step:-0.0331662 <= 0:0.] (at

..\paddle/phi/kernels/funcs/range_function.h:33) [operator < range > error]

当视频生成后好像开始了DAIN的处理,然后出现上述错误,step是负数,看不出来是什么原因造成的

tailangjun commented 1 year ago

我遇到了和你一样的错误

File "parse_weights.py", line 179, in <module>
convert_model(pkl_path, model_path)

InvalidArgumentError: The step should be greater than 0 while start < end.
  [Hint: Expected step > 0, but received step:-3.62781e+19 <= 0:0.] (at /paddle/paddle/phi/kernels/funcs/range_function.h:33)
  [operator < range > error]
wanghx1121 commented 6 months ago

楼主解决了吗,这个问题我也遇到了