search

PaddlePaddle / PaddleOCR

Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, support 80+ languages recognition, provide data annotation and synthesis tools, support training and deployment among server, mobile, embedded and IoT devices)
https://paddlepaddle.github.io/PaddleOCR/
Apache License 2.0
43.76k stars 7.79k forks source link

服务器版本部署paddleocr的cpp推理检测或者识别模型，报错”some trt inputs dynamic shape info not set“， #4903

Closed apple2333cream closed 2 years ago

apple2333cream commented 2 years ago

Error Message Summary:

InvalidArgumentError: some trt inputs dynamic shape info not set, check the INFO log above for more details. [Hint: Expected all_dynamic_shape_set == true, but received all_dynamic_shape_set:0 != true:1.] (at /paddle/paddle/fluid/inference/tensorrt/convert/op_converter.h:245)

andyjiang1116 commented 2 years ago

请问，检测和识别分别用的是哪个模型？

apple2333cream commented 2 years ago

2.2版本的服务器版推理模型

prettyocean85 commented 2 years ago

我遇到过类似的问题。尝试一下关闭分类是否可以，如果只是分类不用能trt是因为分类模型没有设置SetTRTDynamicShapeInfo，分类需要设置一下min_input_shape = {"x": [1, 3, 48, 10]} max_input_shape = {"x": [args.rec_batch_num, 3, 48, 2000]} opt_input_shape = {"x": [args.rec_batch_num, 3, 48, 320]}

mokadevcloud commented 2 years ago

Hi, I am facing the same error. It's quite strange as TensorRT works fine when I run inference in the main process. But when I try to deploy using TorchServe, which loads paddle ocr TextSystem in a subprocess, I have this error. On the other hand, if I set use_tensorrt = False, I don't have any problem, even on TorchServe.

2022-03-01T11:55:49,197 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG - model_name: paddle_ocr, batchSize: 1
2022-03-01T11:55:50,598 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG - Loading PaddleOCR on cuda:0...
2022-03-01T11:55:57,634 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG - Backend worker process died.
2022-03-01T11:55:57,634 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG - Traceback (most recent call last):
2022-03-01T11:55:57,634 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -   File "/home/paddle/env/lib/python3.8/site-packages/ts/model_service_worker.py", line 189, in <module>
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -     worker.run_server()
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -   File "/home/paddle/env/lib/python3.8/site-packages/ts/model_service_worker.py", line 161, in run_server
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -     self.handle_connection(cl_socket)
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -   File "/home/paddle/env/lib/python3.8/site-packages/ts/model_service_worker.py", line 123, in handle_connection
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -     service, result, code = self.load_model(msg)
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -   File "/home/paddle/env/lib/python3.8/site-packages/ts/model_service_worker.py", line 95, in load_model
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -     service = model_loader.load(model_name, model_dir, handler, gpu,
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -   File "/home/paddle/env/lib/python3.8/site-packages/ts/model_loader.py", line 112, in load
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -     initialize_fn(service.context)
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -   File "/tmp/models/4adc412f751f413f91618f8fa1d4d691/base_handler.py", line 44, in initialize
2022-03-01T11:55:57,635 [INFO ] epollEventLoopGroup-5-13 org.pytorch.serve.wlm.WorkerThread - 9002 Worker disconnected. WORKER_STARTED
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -     self.load()
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -   File "/tmp/models/4adc412f751f413f91618f8fa1d4d691/paddle_ocr_handler.py", line 27, in load
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -     text_detector = TextDetector(self.args)
2022-03-01T11:55:57,635 [DEBUG] W-9002-paddle_ocr_1.0 org.pytorch.serve.wlm.WorkerThread - System state is : WORKER_STARTED
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -   File "/home/paddle/env/src/paddleocr/ptools/infer/predict_det.py", line 99, in __init__
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -     utility.create_predictor(args, 'det', logger)
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -   File "/home/paddle/env/src/paddleocr/ptools/infer/utility.py", line 298, in create_predictor
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -     predictor = inference.create_predictor(config)
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG - ValueError: (InvalidArgument) some trt inputs dynamic shape info not set, check the INFO log above for more details.
2022-03-01T11:55:57,635 [INFO ] W-9002-paddle_ocr_1.0-stdout MODEL_LOG -   [Hint: Expected all_dynamic_shape_set == true, but received all_dynamic_shape_set:0 != true:1.] (at /paddle/paddle/fluid/inference/tensorrt/convert/op_converter.h:245)
2022-03-01T11:55:57,635 [DEBUG] W-9002-paddle_ocr_1.0 org.pytorch.serve.wlm.WorkerThread - Backend worker monitoring thread interrupted or backend worker process died.
java.lang.InterruptedException: null
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2056) ~[?:?]
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2133) ~[?:?]
    at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:432) ~[?:?]
    at org.pytorch.serve.wlm.WorkerThread.run(WorkerThread.java:189) [model-server.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
    at java.lang.Thread.run(Thread.java:829) [?:?]
2022-03-01T11:55:57,635 [WARN ] W-9002-paddle_ocr_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: paddle_ocr, error: Worker died.
2022-03-01T11:55:57,635 [DEBUG] W-9002-paddle_ocr_1.0 org.pytorch.serve.wlm.WorkerThread - W-9002-paddle_ocr_1.0 State change WORKER_STARTED -> WORKER_STOPPED
2022-03-01T11:55:57,636 [WARN ] W-9002-paddle_ocr_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9002-paddle_ocr_1.0-stderr
2022-03-01T11:55:57,636 [WARN ] W-9002-paddle_ocr_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9002-paddle_ocr_1.0-stdout
2022-03-01T11:55:57,636 [INFO ] W-9002-paddle_ocr_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9002 in 55 seconds.
2022-03-01T11:55:57,712 [INFO ] W-9002-paddle_ocr_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9002-paddle_ocr_1.0-stdout
2022-03-01T11:55:57,712 [INFO ] W-9002-paddle_ocr_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9002-paddle_ocr_1.0-stderr
mokadevcloud commented 2 years ago

A different error arises when I try to run paddle ocr with TensorRT in a python threading.Thread. As the error happens during the execution of compiled C code, it's virtually impossible to debug. Can you please help?

E0228 16:16:30.301519 102182 helper.h:111] 3: [executionContext.cpp::setBindingDimensions::926] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::setBindingDimensions::926, condition: mOptimizationProfile >= 0 && mOptimizationProfile < mEngine.getNbOptimizationProfiles()
)
E0228 16:16:30.301556 102182 helper.h:111] 3: [executionContext.cpp::getBindingDimensions::978] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::getBindingDimensions::978, condition: mOptimizationProfile >= 0 && mOptimizationProfile < mEngine.getNbOptimizationProfiles()
)
E0228 16:16:30.301573 102182 helper.h:111] 3: [executionContext.cpp::getBindingDimensions::978] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::getBindingDimensions::978, condition: mOptimizationProfile >= 0 && mOptimizationProfile < mEngine.getNbOptimizationProfiles()
)
E0228 16:16:30.301585 102182 helper.h:111] 3: [executionContext.cpp::getBindingDimensions::978] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::getBindingDimensions::978, condition: mOptimizationProfile >= 0 && mOptimizationProfile < mEngine.getNbOptimizationProfiles()
)
E0228 16:16:30.301595 102182 helper.h:111] 3: [executionContext.cpp::getBindingDimensions::978] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::getBindingDimensions::978, condition: mOptimizationProfile >= 0 && mOptimizationProfile < mEngine.getNbOptimizationProfiles()
)
E0228 16:16:30.301605 102182 helper.h:111] 3: [executionContext.cpp::getBindingDimensions::978] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::getBindingDimensions::978, condition: mOptimizationProfile >= 0 && mOptimizationProfile < mEngine.getNbOptimizationProfiles()
)
E0228 16:16:30.301620 102182 helper.h:111] 3: [executionContext.cpp::enqueueInternal::318] Error Code 3: API Usage Error (Parameter check failed at: runtime/api/executionContext.cpp::enqueueInternal::318, condition: mOptimizationProfile >= 0 && mOptimizationProfile < mEngine.getNbOptimizationProfiles()
)

followed by 

Exception in thread TextSpotterPaddle:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/paddle/env/src/pypes/pypes/lib/pipe.py", line 65, in run
    data = self.process(data)
  File "/home/paddle/lib/pypes/text_spotter_paddle.py", line 60, in process
    texts, boxes = predict_text_ocr([
  File "/home/paddle/env/src/paddleocr/ptools/infer/predict.py", line 123, in predict_text_ocr
    dt_boxes, rec_res = text_sys(img)
  File "/home/paddle/env/src/paddleocr/ptools/infer/predict.py", line 64, in __call__
    dt_boxes, elapse = self.text_detector(img)
  File "/home/paddle/env/src/paddleocr/ptools/infer/predict_det.py", line 214, in __call__
    self.predictor.run()
NotImplementedError: In user code:

    File "ptools/export_model.py", line 128, in <module>
      main()
    File "ptools/export_model.py", line 124, in main
      export_single_model(model, arch_config, save_path, logger)
    File "ptools/export_model.py", line 79, in export_single_model
      paddle.jit.save(model, save_path)
    File "<decorator-gen-101>", line 2, in save

    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in __impl__
      return wrapped_func(*args, **kwargs)
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/dygraph/base.py", line 51, in __impl__
      return func(*args, **kwargs)
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/dygraph/jit.py", line 743, in save
      concrete_program = static_func.concrete_program_specify_input_spec(
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 516, in concrete_program_specify_input_spec
      concrete_program, _ = self.get_concrete_program(
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 427, in get_concrete_program
      concrete_program, partial_program_layer = self._program_cache[cache_key]
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 723, in __getitem__
      self._caches[item] = self._build_once(item)
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 709, in _build_once
      concrete_program = ConcreteProgram.from_func_spec(
    File "<decorator-gen-99>", line 2, in from_func_spec

    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/wrapped_decorator.py", line 25, in __impl__
      return wrapped_func(*args, **kwargs)
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/dygraph/base.py", line 51, in __impl__
      return func(*args, **kwargs)
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py", line 662, in from_func_spec
      outputs = static_func(*inputs)
    File "/tmp/tmpvbw5y0g4.py", line 28, in forward
      x = paddle.jit.dy2static.convert_ifelse(self.use_neck, true_fn_1,
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 211, in convert_ifelse
      out = _run_py_ifelse(pred, true_fn, false_fn, true_args, false_args)
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 257, in _run_py_ifelse
      return true_fn(*true_args) if pred else false_fn(*false_args)
    File "/home/paddle/env/src/paddleocr/ppocr/modeling/architectures/base_model.py", line 78, in forward
      x = self.neck(x)
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 917, in __call__
      return self._dygraph_call_func(*inputs, **kwargs)
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/dygraph/layers.py", line 907, in _dygraph_call_func
      outputs = self.forward(*inputs, **kwargs)
    File "/home/paddle/env/src/paddleocr/ppocr/modeling/necks/db_fpn.py", line 92, in forward
      out4 = in4 + F.upsample(
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/nn/functional/common.py", line 681, in upsample
      return interpolate(x, size, scale_factor, mode, align_corners, align_mode,
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/nn/functional/common.py", line 472, in interpolate
      helper.append_op(
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/layer_helper.py", line 43, in append_op
      return self.main_program.current_block().append_op(*args, **kwargs)
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/framework.py", line 3178, in append_op
      op = Operator(
    File "/home/paddle/env/lib/python3.8/site-packages/paddle/fluid/framework.py", line 2224, in __init__
      for frame in traceback.extract_stack():

    UnimplementedError: Input(X) dimension must be 3, 4 or 5, but got dimension = 0 . (at /paddle/paddle/fluid/operators/interpolate_v2_op.cc:395)
      [operator < nearest_interp_v2 > error]
ainndejj11 commented 2 years ago

2.2版本的服务器版推理模型

请问该问题解决了么

Rm1n90 commented 2 years ago

Add config.exp_disable_tensorrt_ops(["transpose2"]) in line 280 in utility.py

andyjiang1116 commented 2 years ago

可以更新一下最新版代码试试