search

isarsoft / yolov4-triton-tensorrt

This repository deploys YOLOv4 as an optimized TensorRT engine to Triton Inference Server
http://www.isarsoft.com
Other
276 stars 63 forks source link

Facing problem to create "engine" #50

Closed audrey-siqueira closed 2 years ago

audrey-siqueira commented 2 years ago

Hi, following the first steps , I did it successfully , without problem :

cd yourworkingdirectoryhere git clone git@github.com:isarsoft/yolov4-triton-tensorrt.git docker run --gpus all -it --rm -v $(pwd)/yolov4-triton-tensorrt:/yolov4-triton-tensorrt nvcr.io/nvidia/tensorrt:21.03-py3

then

cd /yolov4-triton-tensorrt mkdir build cd build cmake .. make

until here no problem.

When I try the following command : ./main with liblayerplugin.so , main and the downloaded yolov4.wts all in the same folder , as the image below:

imagem1

The screen is stuck in the command [Info] Creating model yolov4 , I thought maybe it was the processing time, but it was more than 12 hours like this and nothing happened.

What am I doing wrong? I already tried to change the version of the container but it remained the same.

I would appreciate if someone could help me.

philipp-schmidt commented 2 years ago

Can you share your GPU model and also try to run nvidia-smi while it is optimizing? You should see some load on the GPU.

12 hours is too much.

audrey-siqueira commented 2 years ago

I have a Geforce RTX 2060 .

the output in my host while running /.main inside container is the following:

image

no process is running.

How to proceed?

philipp-schmidt commented 2 years ago

You have a very recent driver version. I didn't even know driver 510 is already available. Maybe there is an issue. You can check if it works with driver 470, and also check whether other containers run, e.g. docker cuda container with nvidia smi. Not sure why it does not list a process.

philipp-schmidt commented 2 years ago

nvidia-smi also does report a few weird stats if you have a closer look. Most things are N/A and your card most likely does not only consume 1 Watt.

audrey-siqueira commented 2 years ago

Actually, because I'm using WSL-2 my driver is 510 and there are these anormal parameters , I'll downgrade to 470 then write you if changes something.

philipp-schmidt commented 2 years ago

I'm not sure about using GPU under WSL2. Especially if Windows is using it at the same time. So this is definitly not an issue with the code. I suggest you install a native linux OS.

audrey-siqueira commented 2 years ago

I could load yolov4 as an .onnx file inside triton server container.

Using .onnx I didnt need the plugin file, loadead without that , but I dont know if it makes any difference.

Do you think that I could use the same client script and procedure in the same way ?

philipp-schmidt commented 2 years ago

You have to try that out, I haven't tested tritons onnx backend. Check layer names in the onnx because they are probably different.