训练问题：PreconditionNotMetError: warp-ctc [version 2] Error in get_workspace_size: invalid value

[jeffzhengye]

用里面数据增强方法得到新数据后混合训练，才会出这个问题。 单独数据集都是好的，但是只也看不出来数据集有什么问题啊。 请教下，可能是什么问题啊？

  File "/home/user/anaconda3/envs/py38/lib/python3.8/site-packages/paddle/fluid/executor.py", line 913, in _run_parallel                                                    tensors = exe.run(fetch_var_names, return_merged)._move_to_list()                                                                                                   RuntimeError: In user code:                                                                                                                                                                                                                                                                                                                         File "train.py", line 93, in <module>                                                                                                                                     main()                                                                                                                                                                File "train.py", line 89, in main                                                                                                                                         train()                                                                                                                                                               File "train.py", line 77, in train                                                                                                                                        ds2_model.train(train_batch_reader=train_batch_reader,                                                                                                                File "/data/yezheng/audio/PaddlePaddle-DeepSpeech/model_utils/model.py", line 217, in train
      train_reader, _, ctc_loss = self.create_network()
    File "/data/yezheng/audio/PaddlePaddle-DeepSpeech/model_utils/model.py", line 130, in create_network
      log_probs, loss = deep_speech_v2_network(audio_data=audio_data,
    File "/data/yezheng/audio/PaddlePaddle-DeepSpeech/model_utils/network.py", line 199, in deep_speech_v2_network
      ctc_loss = paddle.nn.functional.ctc_loss(log_probs=fc, labels=text_data, blank=blank, norm_by_times=True,
    File "/home/user/anaconda3/envs/py38/lib/python3.8/site-packages/paddle/nn/functional/loss.py", line 1087, in ctc_loss
      loss_out = fluid.layers.warpctc(log_probs, labels, blank, norm_by_times,
    File "/home/user/anaconda3/envs/py38/lib/python3.8/site-packages/paddle/fluid/layers/loss.py", line 627, in warpctc
      helper.append_op(
    File "/home/user/anaconda3/envs/py38/lib/python3.8/site-packages/paddle/fluid/layer_helper.py", line 43, in append_op
      return self.main_program.current_block().append_op(*args, **kwargs)
    File "/home/user/anaconda3/envs/py38/lib/python3.8/site-packages/paddle/fluid/framework.py", line 2961, in append_op
      op = Operator(
    File "/home/user/anaconda3/envs/py38/lib/python3.8/site-packages/paddle/fluid/framework.py", line 2039, in __init__
      for frame in traceback.extract_stack():

    PreconditionNotMetError: warp-ctc [version 2] Error in get_workspace_size: invalid value
      [Hint: Expected CTC_STATUS_SUCCESS == status, but received CTC_STATUS_SUCCESS:0 != status:2.] (at /paddle/paddle/fluid/operators/warpctc_op.h:154)
      [operator < warpctc > error]

[yeyupiaoling]

你说的数据增强是指那一个，单纯的数据增强是在训练的时候实现的。

[jeffzhengye]

语音合成方式来增加数据

[yeyupiaoling]

@jeffzhengye 你拉下最新代码试试，可能是因为旧代码没有过滤非常短的音频导致的。

[jeffzhengye]

@yeyupiaoling 按照你所说用最新代码，训练的问题可以解决。但是预测会报错 python infer_server.py --decoding_method ctc_beam_search

使用beam_search 会出现以下错误

[yeyupiaoling]

@jeffzhengye 按照文档：https://github.com/yeyupiaoling/PaddlePaddle-DeepSpeech/blob/master/docs/beam_search.md 重新安装解码器