Open joisonwk opened 1 year ago
解决没?
pip uninstall paddlepaddle-gpu 然后安装2.4.2版本 然后没再出现上述问题了
感谢 解决了, 我之前是看paddlespeech首页Readme提示paddlepaddle>=2.5.0, 我安装的开发版的0.0.0.post117, 按您给的提示问题解决了
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
环境如下 系统:wsl2 + ubuntu 20.04 python版本:3.8 pytorch: 2.0.1 cuda: 11.7 出错运行代码如下,前面stage 0 - 4步运行都成功了
./run.sh --stage 5 --stop-stage 5 finetune... rank: 0, pid: 15672, parent_pid: 15660 multiple speaker fastspeech2! spk_num: 174 samplers done! dataloaders done! vocab_size: 306 W0626 23:15:20.386186 15672 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 7.5, Driver API Version: 12.1, Runtime API Version: 11.7 W0626 23:15:20.387040 15672 gpu_resources.cc:149] device: 0, cuDNN Version: 8.9. I0626 23:15:26.244715 15672 eager_method.cc:143] Warning:: 0D Tensor cannot be used as 'Tensor.numpy()[0]' . In order to avoid this problem, 0D Tensor will be changed to 1D numpy currently, but it's not correct and will be removed in release 2.6. For Tensor contain only one element, Please modify 'Tensor.numpy()[0]' to 'float(Tensor)' as soon as possible, otherwise 'Tensor.numpy()[0]' will raise error in release 2.6. I0626 23:15:26.245553 15672 eager_method.cc:143] Warning:: 0D Tensor cannot be used as 'Tensor.numpy()[0]' . In order to avoid this problem, 0D Tensor will be changed to 1D numpy currently, but it's not correct and will be removed in release 2.6. For Tensor contain only one element, Please modify 'Tensor.numpy()[0]' to 'float(Tensor)' as soon as possible, otherwise 'Tensor.numpy()[0]' will raise error in release 2.6. model done! optimizer done! /home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/nn/layer/layers.py:1896: UserWarning: Skip loading for encoder.embed.1.alpha. encoder.embed.1.alpha receives a shape [1], but the expected shape is []. warnings.warn(f"Skip loading for {key}. " + str(err)) /home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/nn/layer/layers.py:1896: UserWarning: Skip loading for decoder.embed.0.alpha. decoder.embed.0.alpha receives a shape [1], but the expected shape is []. warnings.warn(f"Skip loading for {key}. " + str(err)) /home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/nn/layer/norm.py:776: UserWarning: When training, we now always track global mean and variance. warnings.warn( Exception in main training loop: Variable Shape not match, Variable [ create_parameter_3.w_0_moment1_0 ] need tensor with shape [] but load set tensor with shape [1] Traceback (most recent call last): File "/home/ant/voice/PaddleSpeech/paddlespeech/t2s/training/trainer.py", line 149, in run update() File "/home/ant/voice/PaddleSpeech/paddlespeech/t2s/training/updaters/standard_updater.py", line 110, in update self.update_core(batch) File "/home/ant/voice/PaddleSpeech/paddlespeech/t2s/models/fastspeech2/fastspeech2_updater.py", line 118, in update_core optimizer.step() File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/decorator.py", line 232, in fun return caller(func, *(extras + args), kw) File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/fluid/dygraph/base.py", line 334, in impl return func(*args, *kwargs) File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/decorator.py", line 232, in fun return caller(func, (extras + args), kw) File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in impl return wrapped_func(*args, kwargs) File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/fluid/framework.py", line 462, in impl return func(*args, *kwargs) File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/optimizer/adam.py", line 446, in step optimize_ops = self._apply_optimize( File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/optimizer/optimizer.py", line 1242, in _apply_optimize optimize_ops = self._create_optimization_pass( File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/optimizer/optimizer.py", line 994, in _create_optimization_pass self._create_accumulators( File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/optimizer/adam.py", line 278, in _create_accumulators self._add_moments_pows(p) File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/optimizer/adam.py", line 231, in _add_moments_pows self._add_accumulator(self._moment1_acc_str, p, dtype=acc_dtype) File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/optimizer/optimizer.py", line 799, in _add_accumulator var.set_value(self._accumulators_holder.pop(var_name)) File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/decorator.py", line 232, in fun return caller(func, (extras + args), kw) File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in impl return wrapped_func(*args, kwargs) File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/fluid/framework.py", line 449, in impl return func(*args, kwargs) File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/fluid/dygraph/tensor_patch_methods.py", line 196, in set_value assert self.shape == list( Trainer extensions will try to handle the extension. Then all extensions will finalize.Traceback (most recent call last): File "local/finetune.py", line 269, in
train_sp(train_args, config)
File "local/finetune.py", line 202, in train_sp
trainer.run()
File "/home/ant/voice/PaddleSpeech/paddlespeech/t2s/training/trainer.py", line 198, in run
six.reraise(exc_info)
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/six.py", line 719, in reraise
raise value
File "/home/ant/voice/PaddleSpeech/paddlespeech/t2s/training/trainer.py", line 149, in run
update()
File "/home/ant/voice/PaddleSpeech/paddlespeech/t2s/training/updaters/standard_updater.py", line 110, in update
self.update_core(batch)
File "/home/ant/voice/PaddleSpeech/paddlespeech/t2s/models/fastspeech2/fastspeech2_updater.py", line 118, in update_core
optimizer.step()
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/decorator.py", line 232, in fun
return caller(func, (extras + args), kw)
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/fluid/dygraph/base.py", line 334, in impl
return func(*args, kwargs)
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/decorator.py", line 232, in fun
return caller(func, *(extras + args), kw)
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in impl
return wrapped_func(*args, *kwargs)
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/fluid/framework.py", line 462, in impl
return func(args, kwargs)
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/optimizer/adam.py", line 446, in step
optimize_ops = self._apply_optimize(
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/optimizer/optimizer.py", line 1242, in _apply_optimize
optimize_ops = self._create_optimization_pass(
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/optimizer/optimizer.py", line 994, in _create_optimization_pass
self._create_accumulators(
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/optimizer/adam.py", line 278, in _create_accumulators
self._add_moments_pows(p)
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/optimizer/adam.py", line 231, in _add_moments_pows
self._add_accumulator(self._moment1_acc_str, p, dtype=acc_dtype)
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/optimizer/optimizer.py", line 799, in _add_accumulator
var.set_value(self._accumulators_holder.pop(var_name))
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/decorator.py", line 232, in fun
return caller(func, *(extras + args), kw)
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in impl
return wrapped_func(*args, *kwargs)
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/fluid/framework.py", line 449, in impl
return func(args, kwargs)
File "/home/ant/voice/PaddleSpeech/tools/venv/lib/python3.8/site-packages/paddle/fluid/dygraph/tensor_patch_methods.py", line 196, in set_value
assert self.shape == list(
AssertionError: Variable Shape not match, Variable [ create_parameter_3.w_0_moment1_0 ] need tensor with shape [] but load set tensor with shape [1]
这个需要如何解决呢? 提问前都找了相关的资料都没有解决方法,哪位大神指点下,万分感谢!