enazoe / yolo-tensorrt

TensorRT8.Support Yolov5n,s,m,l,x .darknet -> tensorrt. Yolov4 Yolov3 use raw darknet *.weights and *.cfg fils. If the wrapper is useful to you,please Star it.
MIT License
1.18k stars 313 forks source link

Segmentation fault while converting yolov4-tiny on Jetson Nano #68

Open srikar242 opened 3 years ago

srikar242 commented 3 years ago

Hi..Thanks @enazoe for sharing your work with us. While trying to convert yolov4-tiny into tensorrt, I am getting segmentation fault on nano. And while I was giving yolov4-tiny model, it is trying to load 81 layers which I didn't understand.Basically what I did is, inside the new folder named data created by me, I have put my yolov4-tiny weights and cfg file and changed the paths in samples/sample_detector.cpp file to the respective yolov4-tiny weights and config file. Then from build directory I am running ./yolo-trt. Is there anything wrong in this? Cuda 10.2 , tensorrt 7.1.3.4. `

        Config config_v4_tiny;
    config_v4_tiny.net_type = YOLOV4_TINY;
    config_v4_tiny.detect_thresh = 0.5;
    config_v4_tiny.file_model_cfg = "../data/yolov4-tiny.cfg";
    config_v4_tiny.file_model_weights = "../data/yolov4-tiny.weights";
    config_v4_tiny.calibration_image_list_file_txt = "../configs/calibration_images.txt";
    config_v4_tiny.inference_precison = FP32;
enazoe commented 3 years ago

@srikar242 check the weights file , cfg file and class num

srikar242 commented 3 years ago

Hello @enazoe . I have checked the weights file and config file. They are of yolov4-tiny only. I have trained a custom model with 16 classes. There is no problem with weights and cfg file. Can you mention in 2 to 3 lines what all changes I need to do in the code to convert yolov4-tiny into tensorrt after compilation is done in this repo? Just to verify what I did is right or wrong.

As of now, after compilation I have put my cfg and weights file in a folder and changed paths to them in modules/class_detector.h and in samples/sample_detector.cpp. And then I am just doing ./yolo-trt from build directory. Is this right?

enazoe commented 3 years ago

is right, check the batch size in cfg file ,may be the limite of memory

srikar242 commented 3 years ago

Hello @enazoe . Thanks for your reply. I have checked with some combinations of batch size and subdivisions. batch 1, subdivisions 1 batch 1, subdivisions 16 batch 1, subdivisions 64 batch 4, subdivisions 1 batch 4, subdivisions 16 However I got segmentation fault with all of them. And one more thing which I didn't understand is why till 81 layers are being loaded before segmentation fault comes? Yolov4-tiny doesnt have that many layers.

nano@bai-nano-10:~/yolo-tensorrt$ cd build/
nano@bai-nano-10:~/yolo-tensorrt/build$ ./yolo-trt
File does not exist : ../configs/yolov4-kFLOAT-batch1.engine
Loading pre-trained weights...
Loading complete!
      layer               inp_size            out_size       weightPtr
(1)   conv-bn-mish      3 x 416 x 416      32 x 416 x 416    992   
(2)   conv-bn-mish     32 x 416 x 416      64 x 208 x 208    19680 
(3)   conv-bn-mish     64 x 208 x 208      64 x 208 x 208    24032 
(4)   route                  -             64 x 208 x 208    24032 
(5)   conv-bn-mish     64 x 208 x 208      64 x 208 x 208    28384 
(6)   conv-bn-mish     64 x 208 x 208      32 x 208 x 208    30560 
(7)   conv-bn-mish     32 x 208 x 208      64 x 208 x 208    49248 
(8)   skip             64 x 208 x 208      64 x 208 x 208        - 
(9)   conv-bn-mish     64 x 208 x 208      64 x 208 x 208    53600 
(10)  route                  -            128 x 208 x 208    53600 
(11)  conv-bn-mish    128 x 208 x 208      64 x 208 x 208    62048 
(12)  conv-bn-mish     64 x 208 x 208     128 x 104 x 104    136288
(13)  conv-bn-mish    128 x 104 x 104      64 x 104 x 104    144736
(14)  route                  -            128 x 104 x 104    144736
(15)  conv-bn-mish    128 x 104 x 104      64 x 104 x 104    153184
(16)  conv-bn-mish     64 x 104 x 104      64 x 104 x 104    157536
(17)  conv-bn-mish     64 x 104 x 104      64 x 104 x 104    194656
(18)  skip             64 x 104 x 104      64 x 104 x 104        - 
(19)  conv-bn-mish     64 x 104 x 104      64 x 104 x 104    199008
(20)  conv-bn-mish     64 x 104 x 104      64 x 104 x 104    236128
(21)  skip             64 x 104 x 104      64 x 104 x 104        - 
(22)  conv-bn-mish     64 x 104 x 104      64 x 104 x 104    240480
(23)  route                  -            128 x 104 x 104    240480
(24)  conv-bn-mish    128 x 104 x 104     128 x 104 x 104    257376
(25)  conv-bn-mish    128 x 104 x 104     256 x  52 x  52    553312
(26)  conv-bn-mish    256 x  52 x  52     128 x  52 x  52    586592
(27)  route                  -            256 x  52 x  52    586592
(28)  conv-bn-mish    256 x  52 x  52     128 x  52 x  52    619872
(29)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    636768
(30)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    784736
(31)  skip            128 x  52 x  52     128 x  52 x  52        - 
(32)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    801632
(33)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    949600
(34)  skip            128 x  52 x  52     128 x  52 x  52        - 
(35)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    966496
(36)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1114464
(37)  skip            128 x  52 x  52     128 x  52 x  52        - 
(38)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1131360
(39)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1279328
(40)  skip            128 x  52 x  52     128 x  52 x  52        - 
(41)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1296224
(42)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1444192
(43)  skip            128 x  52 x  52     128 x  52 x  52        - 
(44)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1461088
(45)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1609056
(46)  skip            128 x  52 x  52     128 x  52 x  52        - 
(47)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1625952
(48)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1773920
(49)  skip            128 x  52 x  52     128 x  52 x  52        - 
(50)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1790816
(51)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1938784
(52)  skip            128 x  52 x  52     128 x  52 x  52        - 
(53)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1955680
(54)  route                  -            256 x  52 x  52    1955680
(55)  conv-bn-mish    256 x  52 x  52     256 x  52 x  52    2022240
(56)  conv-bn-mish    256 x  52 x  52     512 x  26 x  26    3203936
(57)  conv-bn-mish    512 x  26 x  26     256 x  26 x  26    3336032
(58)  route                  -            512 x  26 x  26    3336032
(59)  conv-bn-mish    512 x  26 x  26     256 x  26 x  26    3468128
(60)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    3534688
(61)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    4125536
(62)  skip            256 x  26 x  26     256 x  26 x  26        - 
(63)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    4192096
(64)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    4782944
(65)  skip            256 x  26 x  26     256 x  26 x  26        - 
(66)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    4849504
(67)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    5440352
(68)  skip            256 x  26 x  26     256 x  26 x  26        - 
(69)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    5506912
(70)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    6097760
(71)  skip            256 x  26 x  26     256 x  26 x  26        - 
(72)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    6164320
(73)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    6755168
(74)  skip            256 x  26 x  26     256 x  26 x  26        - 
(75)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    6821728
(76)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    7412576
(77)  skip            256 x  26 x  26     256 x  26 x  26        - 
(78)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    7479136
(79)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    8069984
(80)  skip            256 x  26 x  26     256 x  26 x  26        - 
(81)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    8136544
Segmentation fault
enazoe commented 3 years ago

set the net_type as YOLOV4_TINY and confirm the detector->init(config_v4_tiny) and the default model is v4 , the log of your output is the v4

srikar242 commented 3 years ago

Hello @enazoe My net_type is YOLOV4_TINY and detector->init(config_v4_tiny) is also right. But still getting the same log and segmentation fault. In sample_detector.cpp

        Config config_v4_tiny;
        config_v4_tiny.net_type = YOLOV4_TINY;
        config_v4_tiny.detect_thresh = 0.5;
        config_v4_tiny.file_model_cfg = "../data/yolov4-tiny-obj.cfg";
        config_v4_tiny.file_model_weights = "../data/yolov4-tiny.weights";
        config_v4_tiny.calibration_image_list_file_txt = "../configs/calibration_images.txt";
        config_v4_tiny.inference_precison = FP32;

        Config config_v5;
        config_v5.net_type = YOLOV5;
        config_v5.detect_thresh = 0.5;
        config_v5.file_model_cfg = "../configs/yolov5-3.0/yolov5s.cfg";
        config_v5.file_model_weights = "../configs/yolov5-3.0/yolov5s.weights";
        config_v5.inference_precison = FP32;

        std::unique_ptr<Detector> detector(new Detector());
        detector->init(config_v4_tiny);
        cv::Mat image0 = cv::imread("../configs/dog.jpg", cv::IMREAD_UNCHANGED);
    cv::Mat image1 = cv::imread("../configs/person.jpg", cv::IMREAD_UNCHANGED);
    std::vector<BatchResult> batch_res;

In class_detector.h

struct Config
{
    std::string file_model_cfg                  = "data/yolov4-tiny-obj.cfg";

    std::string file_model_weights              = "data/yolov4-tiny.weights";

    float detect_thresh                         = 0.9;

    ModelType   net_type                        = YOLOV4_TINY;

    Precision   inference_precison              = FP32;

    int gpu_id                                  = 0;

    std::string calibration_image_list_file_txt = "configs/calibration_images.txt";

};

class API Detector
{
public:
    explicit Detector();

    ~Detector();

    void init(const Config &config);

    void detect(const std::vector<cv::Mat> &mat_image, std::vector<BatchResult> &vec_batch_result);

private:

    Detector(const Detector &);
    const Detector &operator =(const Detector &);
    class Impl;
    Impl *_impl;
};
enazoe commented 3 years ago

upload the full log,the log you previous up indicate the model you use is yolov4, because the tiny model just have 39 layers

enazoe commented 3 years ago

@srikar242 and the activation of tiny is leaky not mish

srikar242 commented 3 years ago

Hi @enazoe Yes. Thats why I'm also confused. I'm giving tiny weights and its loading v4 weights. I have attached my sample_detector.cpp and class_detector.h files in the zip file for your reference just to see if something is wrong. files.zip

My error log with the values in those files:

nano@bai-nano-10:~/yolo-tensorrt/build$ ./yolo-trt
File does not exist : ../configs/yolov4-kFLOAT-batch1.engine
Loading pre-trained weights...
Loading complete!
      layer               inp_size            out_size       weightPtr
(1)   conv-bn-mish      3 x 416 x 416      32 x 416 x 416    992   
(2)   conv-bn-mish     32 x 416 x 416      64 x 208 x 208    19680 
(3)   conv-bn-mish     64 x 208 x 208      64 x 208 x 208    24032 
(4)   route                  -             64 x 208 x 208    24032 
(5)   conv-bn-mish     64 x 208 x 208      64 x 208 x 208    28384 
(6)   conv-bn-mish     64 x 208 x 208      32 x 208 x 208    30560 
(7)   conv-bn-mish     32 x 208 x 208      64 x 208 x 208    49248 
(8)   skip             64 x 208 x 208      64 x 208 x 208        - 
(9)   conv-bn-mish     64 x 208 x 208      64 x 208 x 208    53600 
(10)  route                  -            128 x 208 x 208    53600 
(11)  conv-bn-mish    128 x 208 x 208      64 x 208 x 208    62048 
(12)  conv-bn-mish     64 x 208 x 208     128 x 104 x 104    136288
(13)  conv-bn-mish    128 x 104 x 104      64 x 104 x 104    144736
(14)  route                  -            128 x 104 x 104    144736
(15)  conv-bn-mish    128 x 104 x 104      64 x 104 x 104    153184
(16)  conv-bn-mish     64 x 104 x 104      64 x 104 x 104    157536
(17)  conv-bn-mish     64 x 104 x 104      64 x 104 x 104    194656
(18)  skip             64 x 104 x 104      64 x 104 x 104        - 
(19)  conv-bn-mish     64 x 104 x 104      64 x 104 x 104    199008
(20)  conv-bn-mish     64 x 104 x 104      64 x 104 x 104    236128
(21)  skip             64 x 104 x 104      64 x 104 x 104        - 
(22)  conv-bn-mish     64 x 104 x 104      64 x 104 x 104    240480
(23)  route                  -            128 x 104 x 104    240480
(24)  conv-bn-mish    128 x 104 x 104     128 x 104 x 104    257376
(25)  conv-bn-mish    128 x 104 x 104     256 x  52 x  52    553312
(26)  conv-bn-mish    256 x  52 x  52     128 x  52 x  52    586592
(27)  route                  -            256 x  52 x  52    586592
(28)  conv-bn-mish    256 x  52 x  52     128 x  52 x  52    619872
(29)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    636768
(30)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    784736
(31)  skip            128 x  52 x  52     128 x  52 x  52        - 
(32)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    801632
(33)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    949600
(34)  skip            128 x  52 x  52     128 x  52 x  52        - 
(35)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    966496
(36)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1114464
(37)  skip            128 x  52 x  52     128 x  52 x  52        - 
(38)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1131360
(39)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1279328
(40)  skip            128 x  52 x  52     128 x  52 x  52        - 
(41)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1296224
(42)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1444192
(43)  skip            128 x  52 x  52     128 x  52 x  52        - 
(44)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1461088
(45)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1609056
(46)  skip            128 x  52 x  52     128 x  52 x  52        - 
(47)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1625952
(48)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1773920
(49)  skip            128 x  52 x  52     128 x  52 x  52        - 
(50)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1790816
(51)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1938784
(52)  skip            128 x  52 x  52     128 x  52 x  52        - 
(53)  conv-bn-mish    128 x  52 x  52     128 x  52 x  52    1955680
(54)  route                  -            256 x  52 x  52    1955680
(55)  conv-bn-mish    256 x  52 x  52     256 x  52 x  52    2022240
(56)  conv-bn-mish    256 x  52 x  52     512 x  26 x  26    3203936
(57)  conv-bn-mish    512 x  26 x  26     256 x  26 x  26    3336032
(58)  route                  -            512 x  26 x  26    3336032
(59)  conv-bn-mish    512 x  26 x  26     256 x  26 x  26    3468128
(60)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    3534688
(61)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    4125536
(62)  skip            256 x  26 x  26     256 x  26 x  26        - 
(63)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    4192096
(64)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    4782944
(65)  skip            256 x  26 x  26     256 x  26 x  26        - 
(66)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    4849504
(67)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    5440352
(68)  skip            256 x  26 x  26     256 x  26 x  26        - 
(69)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    5506912
(70)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    6097760
(71)  skip            256 x  26 x  26     256 x  26 x  26        - 
(72)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    6164320
(73)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    6755168
(74)  skip            256 x  26 x  26     256 x  26 x  26        - 
(75)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    6821728
(76)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    7412576
(77)  skip            256 x  26 x  26     256 x  26 x  26        - 
(78)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    7479136
(79)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    8069984
(80)  skip            256 x  26 x  26     256 x  26 x  26        - 
(81)  conv-bn-mish    256 x  26 x  26     256 x  26 x  26    8136544
Segmentation fault
srikar242 commented 3 years ago

Hello @enazoe . I have solved the issue. I cloned fresh repo and recompiled everything again from beginning .Then it started working. There must have been some problem in compilation. Anyways, thank you so much for giving your time on this issue.

srikar242 commented 3 years ago

Hi @enazoe I have one doubt. Now that I have converted the tiny model into trt engine file, I am trying to run it. In sample_detector.cpp file I see that they are using 2 images,that of a dog.jpg and person.jpg . I commented out all the lines in that file related to displaying an image as I don't want to display. Now when I run trt engine file ,its running continuously for several hundreds of images. On which images is this running? Evencalibration_images.txt file has only paths to 110 images.

nano@bai-nano-10:~/yolo-tensorrt/build$ ./yolo-trt
Loading TRT Engine...
Loading Complete!
detect elasped time:972.444ms
detect elasped time:60.3399ms
detect elasped time:59.3145ms
detect elasped time:61.1872ms
detect elasped time:57.8495ms
detect elasped time:52.8344ms
detect elasped time:52.6705ms
detect elasped time:52.6958ms
detect elasped time:52.7997ms
detect elasped time:52.8762ms
detect elasped time:53.118ms
detect elasped time:52.5891ms
detect elasped time:48.2661ms
detect elasped time:47.8841ms
detect elasped time:48.6153ms
detect elasped time:46.1207ms
detect elasped time:45.8427ms
detect elasped time:44.2754ms
detect elasped time:41.2985ms
detect elasped time:41.5618ms
detect elasped time:41.3914ms
detect elasped time:41.6984ms
detect elasped time:41.3846ms
detect elasped time:38.5885ms
detect elasped time:38.7625ms
detect elasped time:39.064ms
detect elasped time:39.3096ms
detect elasped time:37.0715ms
detect elasped time:36.8794ms
detect elasped time:36.732ms
detect elasped time:36.7905ms
detect elasped time:36.7508ms
detect elasped time:36.8986ms
detect elasped time:37.1645ms
detect elasped time:37.0161ms
detect elasped time:36.7726ms
detect elasped time:37.317ms
detect elasped time:36.6987ms
detect elasped time:36.7003ms
detect elasped time:37.2282ms
detect elasped time:36.7952ms
detect elasped time:36.7859ms
detect elasped time:37.0393ms
detect elasped time:37.1115ms
detect elasped time:36.7304ms
detect elasped time:37.0392ms
detect elasped time:36.9761ms
detect elasped time:37.2046ms
detect elasped time:37.1501ms
detect elasped time:36.9898ms
detect elasped time:36.8315ms
detect elasped time:36.6485ms
detect elasped time:36.7802ms
detect elasped time:37.0701ms
detect elasped time:38.0577ms
detect elasped time:38.0744ms
detect elasped time:37.846ms
detect elasped time:38.1525ms
detect elasped time:37.7737ms
detect elasped time:37.7072ms
detect elasped time:36.492ms
detect elasped time:36.9799ms
detect elasped time:36.6754ms
detect elasped time:36.8814ms
detect elasped time:36.8381ms
detect elasped time:36.6253ms
detect elasped time:37.4199ms
detect elasped time:36.795ms
detect elasped time:36.8896ms
detect elasped time:36.75ms
detect elasped time:36.9622ms
detect elasped time:36.7096ms
detect elasped time:36.7032ms
detect elasped time:37.3897ms
detect elasped time:37.0962ms
detect elasped time:36.9216ms
detect elasped time:37.2951ms
detect elasped time:36.839ms
detect elasped time:36.9443ms

This log goes on continuously until I stop it.