search

jkjung-avt / tensorrt_demos

TensorRT MODNet, YOLOv4, YOLOv3, SSD, MTCNN, and GoogLeNet
https://jkjung-avt.github.io/
MIT License
1.75k stars 547 forks source link

Custom Yolov3 TensorRT Implementation #154

Closed Farjad3253 closed 4 years ago

Farjad3253 commented 4 years ago

Hello! I am a beginner in this field and I have a very basic question that in the code in demo# 4 there is a command line argument : --model yolov3-416. Now I want to ask that whats the contents of this folder. i mean is this a folder which we have to make? And if it is so then what files are to be kept inside this folder?

jkjung-avt commented 4 years ago

When you specify --model yolov3-416, the code will look for this TensorRT engine file: "yolo/yolov3-416.trt". This file is created after you follow the steps: downloading the darknet model, convert darknet model to onnx, and parse onnx to generate the TensorRT engine.

For details, refer to the code pieces below:

    def _load_engine(self):
        TRTbin = 'yolo/%s.trt' % self.model
        with open(TRTbin, 'rb') as f, trt.Runtime(self.trt_logger) as runtime:
            return runtime.deserialize_cuda_engine(f.read())
Farjad3253 commented 4 years ago

I understood that but where to put my yolov3 weights and cfg file in your github repository?

On Mon, 20 Jul 2020 at 13:19, JK Jung notifications@github.com wrote:

When you specify --model yolov3-416, the code will look for this TensorRT engine file: "yolo/yolov3-416.trt". This file is created after you follow the steps: downloading the darknet model, convert darknet model to onnx, and parse onnx to generate the TensorRT engine.

For details, refer to the code pieces below:

- https://github.com/jkjung-avt/tensorrt_demos/blob/master/trt_yolo.py#L102

https://github.com/jkjung-avt/tensorrt_demos/blob/master/utils/yolo.py#L500

https://github.com/jkjung-avt/tensorrt_demos/blob/master/utils/yolo.py#L403

def _load_engine(self):
    TRTbin = 'yolo/%s.trt' % self.model
    with open(TRTbin, 'rb') as f, trt.Runtime(self.trt_logger) as runtime:
        return runtime.deserialize_cuda_engine(f.read())

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jkjung-avt/tensorrt_demos/issues/154#issuecomment-660878581, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUA42PBECV726WWTYTDR5LR4P4XZANCNFSM4PB4D5EA .

jkjung-avt commented 4 years ago

In the "yolo/" directory.

For example, you could name your own custom model files as "yolov3-custom-416.cfg" and "yolov3-custom-416.weights."

Farjad3253 commented 4 years ago

Thanks a lot Sir for responding very quickly. I will run your script and if found any errors then will let you know. Thanks & Regards

On Mon, 20 Jul 2020 at 14:04, JK Jung notifications@github.com wrote:

In the "yolo/" directory.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jkjung-avt/tensorrt_demos/issues/154#issuecomment-660901866, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUA42LMDOOWOTGM57SUDKTR4QCBFANCNFSM4PB4D5EA .

Farjad3253 commented 4 years ago

Hello! Can you please tell me that what is the purpose of using ./download_yolov3.sh in custom TensorRT yolov3 training?

On Mon, 20 Jul 2020 at 17:22, Farjad Haider farjadhaider3253@gmail.com wrote:

Thanks a lot Sir for responding very quickly. I will run your script and if found any errors then will let you know. Thanks & Regards

On Mon, 20 Jul 2020 at 14:04, JK Jung notifications@github.com wrote:

In the "yolo/" directory.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jkjung-avt/tensorrt_demos/issues/154#issuecomment-660901866, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUA42LMDOOWOTGM57SUDKTR4QCBFANCNFSM4PB4D5EA .

jkjung-avt commented 4 years ago

The "download_yolov3.sh" script is used to download the trained yolov3 and yolov3-tiny models, and generate the "yolov3-[288|416|608].cfg", "yolov3-[288|416|608].weights", "yolov3-tiny-[288|416].cfg", and "yolov3-tiny-[288|416].weights" files.

If you have your own custom model, you don't need to run "./download_yolov3.sh".

Farjad3253 commented 4 years ago

Ok thanks. And one more thing that when we call "download_yolov3.sh" then the pre-trained weights and cfg will be downloaded and stored under yolo/ directory? Is it so? Secondly the --model-dir flag is used to specify the directory where the converted onxx file and tensorRT file be saved?

Lastly thanks a lot for helping me and I will let you know if I run into any errors!

On Tue, 21 Jul 2020 at 06:06, JK Jung notifications@github.com wrote:

The "download_yolov3.sh" script is used to download the trained yolov3 and yolov3-tiny models, and generate the "yolov3-[288|416|608].cfg", "yolov3-[288|416|608].weights", "yolov3-tiny-[288|416].cfg", and "yolov3-tiny-[288|416].weights" files.

If you have your own custom model, you don't need to run "./download_yolov3.sh".

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jkjung-avt/tensorrt_demos/issues/154#issuecomment-661517057, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUA42JC42VUF23U5RF4WWLR4TSYVANCNFSM4PB4D5EA .

jkjung-avt commented 4 years ago

What you are asking are very basic questions... The answers could be easily verified by browsing the content of the script and source code.

When you run "./download_yolov3.sh" in the "yolo/" directory, the cfg and weights files would all be saved in the same directory.

When you run "yolo_to_onnx.py" and "onnx_to_tensorrt.py" inside the "yolo/" directory, the onnx and trt files are also saved in the same directory.

Farjad3253 commented 4 years ago

Thanks alot Sir! Sir sorry for the inconvenience as I am a beginner in this field. Sir i will implement your code and will tell you in case of any errors.

On Tue, 21 Jul 2020 at 07:41, JK Jung notifications@github.com wrote:

What you are asking are very basic questions... The answers could be easily verified by browsing the content of the script and source code.

When you run "./download_yolov3.sh" in the "yolo/" directory, the cfg and weights files would all be saved in the same directory.

When you run "yolo_to_onnx.py" and "onnx_to_tensorrt.py" inside the "yolo/" directory, the onnx and trt files are also saved in the same directory.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jkjung-avt/tensorrt_demos/issues/154#issuecomment-661585100, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUA42OWGYWXNLMGOLXIEVDR4T56XANCNFSM4PB4D5EA .

Farjad3253 commented 4 years ago

Hello Sir! I was asking you regarding the download.sh script because in this link :https://jkjung-avt.github.io/trt-yolov3-custom/ of custom yolov3 you have used this script before running the yolo_to_onnx.py script. I was thats why a little bit confused before when I asked you the question that download.sh should be written there. Thank you Sir. Regards!

On Tue, 21 Jul 2020 at 08:42, Farjad Haider farjadhaider3253@gmail.com wrote:

Thanks alot Sir! Sir sorry for the inconvenience as I am a beginner in this field. Sir i will implement your code and will tell you in case of any errors.

On Tue, 21 Jul 2020 at 07:41, JK Jung notifications@github.com wrote:

What you are asking are very basic questions... The answers could be easily verified by browsing the content of the script and source code.

When you run "./download_yolov3.sh" in the "yolo/" directory, the cfg and weights files would all be saved in the same directory.

When you run "yolo_to_onnx.py" and "onnx_to_tensorrt.py" inside the "yolo/" directory, the onnx and trt files are also saved in the same directory.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jkjung-avt/tensorrt_demos/issues/154#issuecomment-661585100, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUA42OWGYWXNLMGOLXIEVDR4T56XANCNFSM4PB4D5EA .

Farjad3253 commented 4 years ago

Can you tell me that this trt_yolo.py can run on live raspberry pi v2 camera feed ?

On Wed, 22 Jul 2020 at 07:22, Farjad Haider farjadhaider3253@gmail.com wrote:

Hello Sir! I was asking you regarding the download.sh script because in this link :https://jkjung-avt.github.io/trt-yolov3-custom/ of custom yolov3 you have used this script before running the yolo_to_onnx.py script. I was thats why a little bit confused before when I asked you the question that download.sh should be written there. Thank you Sir. Regards!

On Tue, 21 Jul 2020 at 08:42, Farjad Haider farjadhaider3253@gmail.com wrote:

Thanks alot Sir! Sir sorry for the inconvenience as I am a beginner in this field. Sir i will implement your code and will tell you in case of any errors.

On Tue, 21 Jul 2020 at 07:41, JK Jung notifications@github.com wrote:

What you are asking are very basic questions... The answers could be easily verified by browsing the content of the script and source code.

When you run "./download_yolov3.sh" in the "yolo/" directory, the cfg and weights files would all be saved in the same directory.

When you run "yolo_to_onnx.py" and "onnx_to_tensorrt.py" inside the "yolo/" directory, the onnx and trt files are also saved in the same directory.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jkjung-avt/tensorrt_demos/issues/154#issuecomment-661585100, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUA42OWGYWXNLMGOLXIEVDR4T56XANCNFSM4PB4D5EA .

jkjung-avt commented 4 years ago

Can you tell me that this trt_yolo.py can run on live raspberry pi v2 camera feed ?

You could reference this post: http://disq.us/p/22rtcid

And modify the code here: https://github.com/jkjung-avt/tensorrt_demos/blob/master/utils/camera.py#L96

Farjad3253 commented 4 years ago

In camera.py there is already code written for the onboard camera on line 96 as you have mentioned. So what I need to modify? Secondly where this camera_tf_trt.py http://disq.us/url?url=http%3A%2F%2Fcamera_tf_trt.py%3ARId6hiahJCxw1BFaDYBVwc-Y78c&cuid=4880657 script is located? I am not able to find it!

On Wed, 22 Jul 2020 at 07:39, JK Jung notifications@github.com wrote:

Can you tell me that this trt_yolo.py can run on live raspberry pi v2 camera feed ?

You could reference this post: http://disq.us/p/22rtcid

And modify the code here: https://github.com/jkjung-avt/tensorrt_demos/blob/master/utils/camera.py#L96

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jkjung-avt/tensorrt_demos/issues/154#issuecomment-662207916, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUA42KVJIUXDCWEOX7J3E3R4ZGNVANCNFSM4PB4D5EA .

Farjad3253 commented 4 years ago

I have successfully converted my custom yolov3 into trt but now please tell me that what I need to run for pi v2 live camera feed detection, I mean I have to run camera.py or camera_tf_trt.py? And hat are arguments I need?

On Wed, 22 Jul 2020 at 07:55, Farjad Haider farjadhaider3253@gmail.com wrote:

In camera.py there is already code written for the onboard camera on line 96 as you have mentioned. So what I need to modify? Secondly where this camera_tf_trt.py http://disq.us/url?url=http%3A%2F%2Fcamera_tf_trt.py%3ARId6hiahJCxw1BFaDYBVwc-Y78c&cuid=4880657 script is located? I am not able to find it!

On Wed, 22 Jul 2020 at 07:39, JK Jung notifications@github.com wrote:

Can you tell me that this trt_yolo.py can run on live raspberry pi v2 camera feed ?

You could reference this post: http://disq.us/p/22rtcid

And modify the code here: https://github.com/jkjung-avt/tensorrt_demos/blob/master/utils/camera.py#L96

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jkjung-avt/tensorrt_demos/issues/154#issuecomment-662207916, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUA42KVJIUXDCWEOX7J3E3R4ZGNVANCNFSM4PB4D5EA .

jkjung-avt commented 4 years ago

If you cannot read/modify python code, I'm not able to help you further.

"camera_tf_trt.py" is here: https://github.com/jkjung-avt/tf_trt_models/blob/master/camera_tf_trt.py (But I don't think you need to reference it...)

Farjad3253 commented 4 years ago

Sir I am able to read the code and understand as well but in camera.py on line 96 there is already the code written for pi v2 camera and Sir the command line arguments mentioned in camera.py does not include any command for piv2 camera. Sir please do not leave me in mid way. You have already helped me a lot and I require your help as my final year project is at stake and I am a beginner in this field.Thank you sir.

On Wed, 22 Jul 2020 at 08:20, JK Jung notifications@github.com wrote:

If you cannot read/modify python code, I'm not able to help you further.

"camera_tf_trt.py" is here: https://github.com/jkjung-avt/tf_trt_models/blob/master/camera_tf_trt.py (But I don't think you need to reference it...)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jkjung-avt/tensorrt_demos/issues/154#issuecomment-662218622, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUA42KMNLBDMLWEL7SXX3TR4ZLIFANCNFSM4PB4D5EA .

Farjad3253 commented 4 years ago

Hello! Sir I have bought a logitech C270 USB webcam. Now sir kindly tell me how can I use his usb webcam for real time yolov3 custom detection?

On Wed, 22 Jul 2020 at 08:27, Farjad Haider farjadhaider3253@gmail.com wrote:

Sir I am able to read the code and understand as well but in camera.py on line 96 there is already the code written for pi v2 camera and Sir the command line arguments mentioned in camera.py does not include any command for piv2 camera. Sir please do not leave me in mid way. You have already helped me a lot and I require your help as my final year project is at stake and I am a beginner in this field.Thank you sir.

On Wed, 22 Jul 2020 at 08:20, JK Jung notifications@github.com wrote:

If you cannot read/modify python code, I'm not able to help you further.

"camera_tf_trt.py" is here: https://github.com/jkjung-avt/tf_trt_models/blob/master/camera_tf_trt.py (But I don't think you need to reference it...)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jkjung-avt/tensorrt_demos/issues/154#issuecomment-662218622, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUA42KMNLBDMLWEL7SXX3TR4ZLIFANCNFSM4PB4D5EA .

jkjung-avt commented 4 years ago 
$ python3 trt_yolo.py --usb --vid 0 --model yolov3-custom-416
Farjad3253 commented 4 years ago

Thank You very much sir I will run the code and will let you know in case of any errors!

On Thu, 23 Jul 2020 at 10:16, JK Jung notifications@github.com wrote:

$ python3 trt_yolo.py --usb --vid 0 --model yolov3-custom-416

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jkjung-avt/tensorrt_demos/issues/154#issuecomment-662822478, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUA42PJJMJJVW4YCHEQDPDR47BTHANCNFSM4PB4D5EA .

Farjad3253 commented 4 years ago

getting this error :

farjad@farjad-desktop:~/project/tensorrt_demos$ python3 trt_yolo.py --usb --vid 0 --model yolov3-custom-416

[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (933) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1 [TensorRT] WARNING: Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors. Traceback (most recent call last): File "trt_yolo.py", line 116, in main() File "trt_yolo.py", line 108, in main loop_and_detect(cam, trt_yolo, conf_th=0.3, vis=vis) File "trt_yolo.py", line 61, in loop_and_detect boxes, confs, clss = trt_yolo.detect(img, conf_th) File "/home/farjad/project/tensorrt_demos/utils/yolo.py", line 532, in detect for output, shape in zip(trt_outputs, self.output_shapes)] File "/home/farjad/project/tensorrt_demos/utils/yolo.py", line 532, in

for output, shape in zip(trt_outputs, self.output_shapes)] ValueError: cannot reshape array of size 3042 into shape (1,255,13,13) ---------------------------------------------------------------------------------------------------------------------- Kindly help me resolve the issue! I am using a Logitech USB webcam. On Thu, 23 Jul 2020 at 10:41, Farjad Haider wrote: > Thank You very much sir > I will run the code and will let you know in case of any errors! > > On Thu, 23 Jul 2020 at 10:16, JK Jung wrote: > >> $ python3 trt_yolo.py --usb --vid 0 --model yolov3-custom-416 >> >> — >> You are receiving this because you authored the thread. >> Reply to this email directly, view it on GitHub >> , >> or unsubscribe >> >> . >> >
jkjung-avt commented 4 years ago

I think you just need to add "--category_num 1" into the command.

Farjad3253 commented 4 years ago

Thanks a lot @jkjung. Finally the code is working fine when I used the categoty_num 1 command but the label is displayed on video as CSLO while my data had only one class with the label 'FOD' but the actual label is not displayed? Any idea how to resolve this issue?

On Fri, 24 Jul 2020 at 06:02, JK Jung notifications@github.com wrote:

I think you just need to add "--category_num 1" into the command.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jkjung-avt/tensorrt_demos/issues/154#issuecomment-663303764, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUA42PK646CR726RRWDTXLR5DMQVANCNFSM4PB4D5EA .

jkjung-avt commented 4 years ago

I thought you said you can read and understand python code??

Refer to: https://github.com/jkjung-avt/tensorrt_demos/blob/031bc4e4da2b0a7f85dd221802fcd2b595f269cc/utils/yolo_classes.py#L104

For example, you could replace this line:

    cls_dict = get_cls_dict(args.category_num)

with the following

    cls_dict = {0: 'FOD'}
Farjad3253 commented 4 years ago

Thank you so much for your help. Regards!

On Fri, 24 Jul 2020 at 07:49, JK Jung notifications@github.com wrote:

I thought you said you can read and understand python code??

Refer to: https://github.com/jkjung-avt/tensorrt_demos/blob/031bc4e4da2b0a7f85dd221802fcd2b595f269cc/utils/yolo_classes.py#L104

For example, you could replace this line https://github.com/jkjung-avt/tensorrt_demos/blob/031bc4e4da2b0a7f85dd221802fcd2b595f269cc/trt_yolo.py#L90 :

cls_dict = get_cls_dict(args.category_num)

with the following

cls_dict = {0: 'FOD'}

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jkjung-avt/tensorrt_demos/issues/154#issuecomment-663324412, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMUA42JL5RFHT44IWX44DULR5DZDDANCNFSM4PB4D5EA .

Farjad3253 commented 4 years ago

Hello! I want to ask that how can we change the threshold value of detection in case of custom yolov4 tensorrt implementation?