yeyupiaoling / PPASR

基于PaddlePaddle实现端到端中文语音识别,从入门到实战,超简单的入门案例,超实用的企业项目。支持当前最流行的DeepSpeech2、Conformer、Squeezeformer模型
Apache License 2.0
817 stars 129 forks source link

GPU预测时出错,Aborted at 1637208819 (unix time) try "date -d @1637208819" if you are using GNU date #11

Closed zhiweinet closed 2 years ago

zhiweinet commented 3 years ago

老师,您好。

前序步骤【数据准备】、【训练模型】、【执行评估】、【导出模型】均正常通过。

但是在【快速预测】时,执行:python infer_path.py --wav_path=./dataset/test.wav 后,出现以下错误:

root@pp:~/PPASR# python export_model.py --resume_model=models/deepspeech2/epoch_50/
/usr/local/lib/python3.7/dist-packages/numba/types/__init__.py:110: DeprecationWarning: `np.long` is a deprecated alias for `np.compat.long`. To silence this warning, use `np.compat.long` by itself. In the likely event your code does not need to work on Python 2 you can use the builtin `int` for which `np.compat.long` is itself an alias. Doing this will not modify any behaviour and is safe. When replacing `np.long`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  long_ = _make_signed(np.long)
/usr/local/lib/python3.7/dist-packages/numba/types/__init__.py:111: DeprecationWarning: `np.long` is a deprecated alias for `np.compat.long`. To silence this warning, use `np.compat.long` by itself. In the likely event your code does not need to work on Python 2 you can use the builtin `int` for which `np.compat.long` is itself an alias. Doing this will not modify any behaviour and is safe. When replacing `np.long`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  ulong = _make_unsigned(np.long)
/usr/local/lib/python3.7/dist-packages/librosa/cache.py:49: DeprecationWarning: The 'cachedir' attribute has been deprecated in version 0.12 and will be remo
root@pp:~/PPASR# python infer_path.py --wav_path=./dataset/test.wav
/usr/local/lib/python3.7/dist-packages/numba/types/__init__.py:110: DeprecationWarning: `np.long` is a deprecated alias for `np.compat.long`. To silence this warning, use `np.compat.long` by itself. In the likely event your code does not need to work on Python 2 you can use the builtin `int` for which `np.compat.long` is itself an alias. Doing this will not modify any behaviour and is safe. When replacing `np.long`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  long_ = _make_signed(np.long)
/usr/local/lib/python3.7/dist-packages/numba/types/__init__.py:111: DeprecationWarning: `np.long` is a deprecated alias for `np.compat.long`. To silence this warning, use `np.compat.long` by itself. In the likely event your code does not need to work on Python 2 you can use the builtin `int` for which `np.compat.long` is itself an alias. Doing this will not modify any behaviour and is safe. When replacing `np.long`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  ulong = _make_unsigned(np.long)
/usr/local/lib/python3.7/dist-packages/librosa/cache.py:49: DeprecationWarning: The 'cachedir' attribute has been deprecated in version 0.12 and will be removed in version 0.14.
Use os.path.join(memory.location, 'joblib') attribute instead.
  if self.cachedir is not None and self.level >= level:
/usr/local/lib/python3.7/dist-packages/librosa/cache.py:49: DeprecationWarning: The 'cachedir' attribute has been deprecated in version 0.12 and will be removed in version 0.14.
Use os.path.join(memory.location, 'joblib') attribute instead.
  if self.cachedir is not None and self.level >= level:
-----------  Configuration Arguments -----------
alpha: 1.2
beam_size: 10
beta: 0.35
cutoff_prob: 1.0
cutoff_top_n: 40
decoder: ctc_greedy
is_long_audio: False
lang_model_path: lm/zh_giga.no_cna_cmn.prune01244.klm
model_dir: models/deepspeech2/infer/
to_an: True
use_gpu: True
vocab_path: dataset/vocabulary.txt
wav_path: ./dataset/test.wav
------------------------------------------------

--------------------------------------
C++ Traceback (most recent call last):
--------------------------------------
0   paddle::framework::SignalHandle(char const*, int)
1   paddle::platform::GetCurrentTraceBackString[abi:cxx11]()

----------------------
Error Message Summary:
----------------------
FatalError: `Segmentation fault` is detected by the operating system.
  [TimeInfo: *** Aborted at 1637207454 (unix time) try "date -d @1637207454" if you are using GNU date ***]
  [SignalInfo: *** SIGSEGV (@0x0) received by PID 10680 (TID 0x7f258fdb6740) from PID 0 ***]

Segmentation fault

环境如下:

系统:Ubuntu  18.04 64位
显卡:NVIDIA P100 (单卡)
驱动:10.2.89
内存:60G
Python:3.7.5
PaddlePaddle:2.1.3 (PIP安装)
项目:PPASR
模型:thchs_30(34小时) 
noise模型:无

NVIDIA信息如下:

root@pp:~/PPASR# nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Oct_23_19:24:38_PDT_2019
Cuda compilation tools, release 10.2, V10.2.89
root@pp:~/PPASR# nvidia-smi
Thu Nov 18 12:05:23 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.91.03    Driver Version: 460.91.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla P100-PCIE...  On   | 00000000:00:08.0 Off |                    0 |
| N/A   27C    P0    26W / 250W |      0MiB / 16280MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

请问老师,我在项目:“PaddlePaddle-DeepSpeech”和“PPASR"使用【GPU】预测均遇到该问题,使用【CPU】预测正常。在训练时,使用【GPU】也是正常。

请问这个问题是什么原因导致的?麻烦可否告知如何解决?

感谢您百忙这中抽空审阅,谢谢。

yeyupiaoling commented 3 years ago

我实在看不出什么问题,报错的有效消息比较少。你说的使用CPU预测正常,只有使用GPU预测报错,那可能使用你安装的Paddle Inference使用有问题,你可以重装一下,或者使用conda安装一次。

wangjialin97 commented 1 year ago

请问解决了吗