PaddlePaddle / PaddleDetection

Object Detection toolkit based on PaddlePaddle. It supports object detection, instance segmentation, multiple object tracking and real-time multi-person keypoint detection.
Apache License 2.0
12.82k stars 2.89k forks source link

paddleocr ser模型推理报错:../paddle/phi/kernels/funcs/elementwise_functor.h:545 Assertion `b != 0` failed. InvalidArgumentError: Integer division by zero encountered in (floor) divide. Please check the input value. #6474

Closed jackliu1111 closed 2 years ago

jackliu1111 commented 2 years ago

问题确认 Search before asking

请提出你的问题 Please ask your question

运行代码为PaddleOCR-release-2.5 的infer_vqa_token_ser.py,VQA推理,环境为 python 3.9.7 cuda 11.6 cudnn 8.2 paddle-bfloat 0.1.7 paddle2onnx 0.9.8 paddlefsl 1.1.0 paddlehub 2.2.0 paddlenlp 2.3.4 paddlepaddle-gpu 2.3.1.post112

在执行该代码段后并不会生成对应的tensor, visual_bbox_x = (paddle.arange( 0, 1000 * (self.config["image_feature_pool_shape"][1] + 1), 1000, dtype=bbox.dtype, ) // self.config["image_feature_pool_shape"][1])

而是会有大量的被除数是0的报错 Error: ../paddle/phi/kernels/funcs/elementwise_functor.h:545 Assertion b != 0 failed. InvalidArgumentError: Integer division by zero encountered in (floor) divide. Please check the input value. Error: ../paddle/phi/kernels/funcs/elementwise_functor.h:545 Assertion b != 0 failed. InvalidArgumentError: Integer division by zero encountered in (floor) divide. Please check the input value. ... Error: ../paddle/phi/kernels/funcs/elementwise_functor.h:545 Assertion b != 0 failed. InvalidArgumentError: Integer division by zero encountered in (floor) divide. Please check the input value.

Traceback (most recent call last):
  File "C:\Users\Administrator\Desktop\PaddleOCR-release-2.5\tools\infer_vqa_token_ser.py", line 128, in <module>
    result, _ = ser_engine(img_path)
  File "C:\Users\Administrator\Desktop\PaddleOCR-release-2.5\tools\infer_vqa_token_ser.py", line 100, in __call__
    preds = self.model(batch)
  File "E:\python3.9.7\yolo5_env\lib\site-packages\paddle\fluid\dygraph\layers.py", line 930, in __call__
    return self._dygraph_call_func(*inputs, **kwargs)
  File "E:\python3.9.7\yolo5_env\lib\site-packages\paddle\fluid\dygraph\layers.py", line 915, in _dygraph_call_func
    outputs = self.forward(*inputs, **kwargs)
  File "C:\Users\Administrator\Desktop\PaddleOCR-release-2.5\ppocr\modeling\architectures\base_model.py", line 79, in forward
    x = self.backbone(x)
  File "E:\python3.9.7\yolo5_env\lib\site-packages\paddle\fluid\dygraph\layers.py", line 930, in __call__
    return self._dygraph_call_func(*inputs, **kwargs)
  File "E:\python3.9.7\yolo5_env\lib\site-packages\paddle\fluid\dygraph\layers.py", line 915, in _dygraph_call_func
    outputs = self.forward(*inputs, **kwargs)
  File "C:\Users\Administrator\Desktop\PaddleOCR-release-2.5\ppocr\modeling\backbones\vqa_layoutlm.py", line 121, in forward
    x = self.model(
  File "E:\python3.9.7\yolo5_env\lib\site-packages\paddle\fluid\dygraph\layers.py", line 930, in __call__
    return self._dygraph_call_func(*inputs, **kwargs)
  File "E:\python3.9.7\yolo5_env\lib\site-packages\paddle\fluid\dygraph\layers.py", line 915, in _dygraph_call_func
    outputs = self.forward(*inputs, **kwargs)
  File "E:\python3.9.7\yolo5_env\lib\site-packages\paddlenlp\transformers\layoutxlm\modeling.py", line 998, in forward
    outputs = self.layoutxlm(
  File "E:\python3.9.7\yolo5_env\lib\site-packages\paddle\fluid\dygraph\layers.py", line 930, in __call__
    return self._dygraph_call_func(*inputs, **kwargs)
  File "E:\python3.9.7\yolo5_env\lib\site-packages\paddle\fluid\dygraph\layers.py", line 915, in _dygraph_call_func
    outputs = self.forward(*inputs, **kwargs)
  File "E:\python3.9.7\yolo5_env\lib\site-packages\paddlenlp\transformers\layoutxlm\modeling.py", line 874, in forward
    visual_bbox = paddle.stack(
  File "E:\python3.9.7\yolo5_env\lib\site-packages\paddle\tensor\manipulation.py", line 903, in stack
    return layers.stack(x, axis, name)
  File "E:\python3.9.7\yolo5_env\lib\site-packages\paddle\fluid\layers\nn.py", line 10397, in stack
    return _C_ops.stack(x, 'axis', axis)
OSError: (External) CUDA error(719), unspecified launch failure.
  [Hint: 'cudaErrorLaunchFailure'. An exception occurred on the device while executing a kernel. Common causes include dereferencing an invalid device pointerand accessing out of bounds shared memory. Less common cases can be s
ystem specific - more information about these cases canbe found in the system specific user guide. This leaves the process in an inconsistent state and any further CUDA work willreturn the same error. To continue using CUDA, the process must be terminated and relaunched.] (at ..\paddle\phi\backends\gpu\cuda\cuda_info.cc:251)
  [operator < stack > error]

通过对该代码变量的查看,发现是整除过程的to_string.py出现了问题

y = self.config["image_feature_pool_shape"][1] x = paddle.arange(0,1000 * (self.config["image_feature_pool_shape"][1] + 1),1000,dtype=bbox.dtype,) print('y:',y) print('x:',x) print(x/y) print(x//y)

以下是结果: y: 7 x: Tensor(shape=[8], dtype=int64, place=Place(gpu:0), stop_gradient=True, [0 , 1000, 2000, 3000, 4000, 5000, 6000, 7000]) Tensor(shape=[8], dtype=float32, place=Place(gpu:0), stop_gradient=True, [0. , 142.85714722 , 285.71429443 , 428.57144165 ,571.42858887 , 714.28576660 , 857.14288330 , 1000.00006104]) Error: ../paddle/phi/kernels/funcs/elementwise_functor.h:545 Assertion b != 0 failed. InvalidArgumentError: Integer division by zero encountered in (floor) divide. Please check the input value. Error: ../paddle/phi/kernels/funcs/elementwise_functor.h:545 Assertion b != 0 failed. InvalidArgumentError: Integer division by zero encountered in (floor) divide. Please check the input value. ... Error: ../paddle/phi/kernels/funcs/elementwise_functor.h:545 Assertion b != 0 failed. InvalidArgumentError: Integer division by zero encountered in (floor) divide. Please check the input value.

  File "E:\python3.9.7\yolo5_env\lib\site-packages\paddlenlp\transformers\layoutxlm\modeling.py", line 856, in forward
    print(x//y)
  File "E:\python3.9.7\yolo5_env\lib\site-packages\paddle\fluid\dygraph\varbase_patch_methods.py", line 617, in __str__
    return to_string(self)
  File "E:\python3.9.7\yolo5_env\lib\site-packages\paddle\tensor\to_string.py", line 238, in to_string
    np_var = var.numpy()
OSError: (External) CUDA error(719), unspecified launch failure.
  [Hint: 'cudaErrorLaunchFailure'. An exception occurred on the device while executing a kernel. Common causes include dereferencing an invalid device pointerand accessing out of bounds shared memory. Less common cases can be s
ystem specific - more information about these cases canbe found in the system specific user guide. This leaves the process in an inconsistent state and any further CUDA work willreturn the same error. To continue using CUDA, the process must be terminated and relaunched.] (at ..\paddle\phi\backends\gpu\cuda\cuda_info.cc:258)
nemonameless commented 2 years ago

请前往PaddleOCR提问谢谢 https://github.com/PaddlePaddle/PaddleOCR/issues/new/choose