triton-inference-server / paddlepaddle_backend

BSD 3-Clause "New" or "Revised" License
32 stars 6 forks source link

Can not use disenable_trt_tune option #19

Open nhhviet98 opened 1 year ago

nhhviet98 commented 1 year ago

I want to use TRT and set disenable_trt_tune option is True but get an exception below:

unknown parameter 'disenable_trt_tune` is provided for GPU execution accelerator config. Available choices are [precision, min_graph_size, workspace_size, max_batch_size, enable_tensorrt_oss, is_dynamic]

But i saw that this option available here.


This is Docker image base and config.pbtxt for execution accelerators with tensorrt:

  1. Docker image

    https://hub.docker.com/r/paddlepaddle/triton_paddle

  2. Configs.pbtxt

optimization {
  execution_accelerators {
    gpu_execution_accelerator : [
      {
        name : "tensorrt"
        parameters { key: "precision" value: "trt_fp16" }
        parameters { key: "min_graph_size" value: "30" }
        parameters { key: "max_batch_size" value: "32" }
        parameters { key: "workspace_size" value: "1073741824" }
        parameters { key: "is_dynamic" value: "1" }
        parameters { key: "disenable_trt_tune" value: "1" }
      }
    ]
  }
}

Please help me asps, thank you !!! 🥺️🥺️🥺️

JasonChao-yue commented 1 year ago

@nhhviet98 My friend, could you send me a Docker image package? Thank you very much!

nhhviet98 commented 1 year ago

@nhhviet98 My friend, could you send me a Docker image package? Thank you very much!

I've used the official image from here

JasonChao-yue commented 1 year ago

@nhhviet98 My friend, could you send me a Docker image package? Thank you very much!

I've used the official image from here

I have tried this method. It may fail to pull the mirror image due to network reasons.