YOLOv4-tiny model returns no boxes after RT optimization

[Seioch]

I optimized a Yolov4-tiny model, however the optimized model now returns no bounding boxes. Before optimization, my tiny model did return bounding boxes upon testing. Any ideas on what could have gone wrong? The code below was able to optimize and test a standard YOLOv4 model with FP16/FP32 and batch size of 4.

My program to optimize the model:

int main(int argc, char* argv[]) {
    if(argc < 3){
        FatalError("Not enough parameters, must provide path to yolov4 layers, cfg, names");
    }
    std::string rt_file_name = "yolov4_tiny";
    std::string bin_path  = std::string(argv[1]);
    std::vector<std::string> input_bins = {
        bin_path + "/layers/input.bin"
    };
    std::vector<std::string> output_bins = {
        bin_path + "/debug/layer30_out.bin",
        bin_path + "/debug/layer37_out.bin"
        bin_path + "/debug/layer44_out.bin"
    };
    std::string wgs_path  = bin_path + "/layers";
    std::string cfg_path  = std::string(argv[2]);
    std::string name_path = std::string(argv[3]);
    // downloadWeightsifDoNotExist(input_bins[0], bin_path, "https://cloud.hipert.unimore.it/s/d97CFzYqCPCp5Hg/download");

    // parse darknet network
    tk::dnn::Network *net = tk::dnn::darknetParser(cfg_path, wgs_path, name_path);
    net->print();

    //convert network to tensorRT
    tk::dnn::NetworkRT *netRT = new tk::dnn::NetworkRT(net, net->getNetworkRTName(rt_file_name.c_str()));

    int ret = testInference(input_bins, output_bins, net, netRT);
    net->releaseLayers();
    delete net;
    delete netRT;
    return ret;
}

How I tested my RT model for bounding boxes:

#ifndef TKDNN_WRAPPER
#define TKDNN_WRAPPER

#include <iostream>
#include <signal.h>
#include <stdlib.h>
#include <unistd.h>
#include <mutex>
#include <math.h>       /* ceil */

// opencv stuff should probably stay for image loading for debugging only
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>

#include "tkDNN/Yolo3Detection.h"

extern "C++" {
    #include "tkdnn.h" // tkdnn core libraries
    #include "tkDNN/DarknetParser.h" // darknet parser utils
}

// When compiling this wrapper, if your OPENCV2 is compiled with the contrib from opencv4, comment this in for GPU accelerated resize
// #define OPENCV_CUDACONTRIB //if OPENCV has been compiled with CUDA and contrib.

#ifdef OPENCV_CUDACONTRIB
#include <opencv2/cudawarping.hpp>
#include <opencv2/cudaarithm.hpp>
#endif

// This code was adapted from DetectionNN.h
// Note 1: The word "frame" and "image" are used iterchangably in this file. Both refer to an image file.

std::vector<cv::Mat> loadImagesFromList(std::string listPath) {
    cv::Mat image;
    std::vector<cv::Mat> loaded_images;

    std::ifstream input_stream;
    input_stream.open(listPath);
    std::string line;

    while( getline(input_stream, line) ){
        image = cv::imread(line, cv::IMREAD_COLOR);   // Read the file
        if(! image.data ) {                             // Check for invalid input
            FatalError("Image not valid")
        }
        loaded_images.push_back(image);
    }
    return loaded_images;
}

// Write performance to file
void dumpPerformanceStats(tk::dnn::DetectionNN* detNN) {
    std::ofstream outputfile;
    std::cout << "Writing " << detNN->stats.size() << " performance numbers to performance.csv\n";
    outputfile.open("performance.csv");
    for(int i=0; i<detNN->stats.size(); i++){
        outputfile << detNN->stats[i] << ",\n";
    }
    outputfile.close();
}

int main(int argc, char* argv[]){
    if(argc < 4){
        FatalError("Not enough parameters, must provide in order: path to yolov4 rt file, text file with list of images, batch size, number of classes");
    }
    // Set batch size from command lines
    float batch_size = atoi(argv[3]);

    // Initialize Yolo detection objects
    tk::dnn::Yolo3Detection yolo;
    tk::dnn::DetectionNN* network = &yolo;
    std::vector<cv::Mat> dnnInput;
    std::vector<cv::Mat> inference_batch;

    // Initialize the network. Init takes in three parameters: string path to .rt file, number of classes, batch_size
    network->init(argv[1], atoi(argv[4]), batch_size);
    std::cout << "Yolo initialized" << std::endl;

    // Load in images from a text file. Regardless of what happens, your desired images you want to infer on needs to
    // get into a vector of cv::Mat 
    dnnInput = loadImagesFromList(argv[2]);  // Add the image to a vector of frames 

    std::cout << "Loaded " << dnnInput.size() << " images from file " << argv[2] << std::endl;

    // Pop the first 4 items off the loaded images up to a batch_size, then feed batch into NN
    float dnnInputSize = dnnInput.size();
    int num_batches = std::ceil(dnnInputSize/batch_size); // Calculate the number of batches, rounded up
    std::cout << "Number of batches: " << num_batches << std::endl;
    for (int i = 0; i < num_batches; i++ ) {
        // inference_batch is a vector of size batch_size containing cv::mats that we will infer on
        inference_batch.clear(); // clear out the last batch
        for (int b = batch_size*i; b < batch_size*(i+1); b++) {
            if(b < dnnInput.size()) { // Do not add if we cannot make a full batch
                inference_batch.push_back(dnnInput[b].clone());
            }
        }

        std::cout << "Inferring on batch #" << i << " containing " << inference_batch.size() << " images\n";

        // do the inference on inference_batch. 
        network->update(inference_batch, inference_batch.size()); 

        // The raw bounding boxes are saved as a vector of vectors. 
        // The outer vector is a vector of bounding boxes of each batch, indexed by the batch you put in (e.g. index 0 was the first batch
        // you ran network->update on, index 1 is the second, etc.)
        // The inner vector is a vector of tkdnn boxes, representing the bounding boxes per image in the batch. 
        // For example: box[0][0] is batch #0's bounding box for image 1 out of 4
        std::vector<std::vector<tk::dnn::box>> boxes = network->getRawBoundingBoxes();

        // test printout for number of boxes
        std::cout << "Number of BBs: " << boxes[0].size() << std::endl;
        //for (int i = 0; i < boxes[0].size(); i++) {
        for (int i = 0; i < 1; i++) {
            std::cout << "Box 1 bbox " << i << " scaled contents: \n";
            boxes[0][i].print();
        }
    }
    // need to periodically clear out this vector 
    network->batchDetected.clear();

    // std::cout << "Inferred in " << *network->stats.begin() << " milliseconds\n";
    dumpPerformanceStats(network);
}

#endif /* TKDNN_WRAPPER*/

[B1ackg0d]

Hi,@Seioch.Did you solve this problem?I think I also encountered this problem.

[mive93]

Hi @Seioch,

could you explain better? It was working on darknet and there you could see the boxes? Do you use the correct config file and names file? Have you tried using the demo?

[Seioch]

Hi @mive93

I successfully trained a model using darknet, and verified that it works using darknet's test utility. Then I exported the weights using the darknet export utility on the README, and then optimized it with tkdnn using the code I put in my original post.

I have also tried using the default demo to do the optimization, however no difference was made.

[mive93]

I see. Does the creation of the rt file work properly? Do you get all OKs?

[Seioch]

Yes, all OKs are printed when creating RT files with either FP16 or FP32

[mive93]

And, sorry if I ask again, but do you use the correct config file and names file?

[alexkokh]

Could be an issue with exporting weights from darknet. The easiest way to check this is to compare your exported layer file sizes with the sizes of the provided sample layer files.

[mive93]

Closing for inactivity. Feel free to reopen.