Example cpp code for storing mot challenge format for mota, motp measurement...

[YoungjaeDev]

Hello! In order to tune the appropriate value for tracker.yaml in the future, it will be saved in the format suggested by mot challenge Through the monitor_action you developed, the following values are stored in order, and temporarily saved in 'pred.txt' What do you think about these tasks?

<frame>, <id>, <bb_left>, <bb_top>, <bb_width>, <bb_height>, <conf>, <x>, <y>, <z>

/*
The MIT License

Copyright (c) 2021, Prominence AI, Inc.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, an/opt/prominenceai/deepstream-services-library/examples/cpp/1rtsp_yolov5_dcf_osd_window.cppd/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in-
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
*/

/*
https://github.com/JonathonLuiten/TrackEval/blob/master/docs/MOTChallenge-format.txt

File Format

Please submit your results as a single .zip file. The results for each sequence must be stored in a separate .txt file in the archive's root folder. The file name must be exactly like the sequence name (case sensitive).

The file format should be the same as the ground truth file, which is a CSV text-file containing one object instance per line. Each line must contain 10 values:

<frame>, <id>, <bb_left>, <bb_top>, <bb_width>, <bb_height>, <conf>, <x>, <y>, <z>
The conf value contains the detection confidence in the det.txt files. For the ground truth, it acts as a flag whether the entry is to be considered. A value of 0 means that this particular instance is ignored in the evaluation, while any other value can be used to mark it as active. For submitted results, all lines in the .txt file are considered. The world coordinates x,y,z are ignored for the 2D challenge and can be filled with -1. Similarly, the bounding boxes are ignored for the 3D challenge. However, each line is still required to contain 10 values.

All frame numbers, target IDs and bounding boxes are 1-based. Here is an example:

Tracking with bounding boxes
(MOT15, MOT16, MOT17, MOT20)
  1, 3, 794.27, 247.59, 71.245, 174.88, -1, -1, -1, -1
  1, 6, 1648.1, 119.61, 66.504, 163.24, -1, -1, -1, -1
  1, 8, 875.49, 399.98, 95.303, 233.93, -1, -1, -1, -1
  ...
*/

#include <iostream>
#include <glib.h>
#include <vector>
#include <fstream>

#include "DslApi.h"

static const std::wstring primary_infer_config_file(L"/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_infer_primary_nano.txt");

static const std::wstring primary_model_engine_file(L"/opt/nvidia/deepstream/deepstream/samples/models/Primary_Detector_Nano/resnet10.caffemodel_b8_gpu0_fp16.engine");

static const std::wstring tracker_config_file(L"/opt/nvidia/deepstream/deepstream/samples/configs/deepstream-app/config_tracker_NvDCF_max_perf.yml");

int tile_width = 1280;
int tile_height = 720;
int sink_width = 1280;
int sink_height = 720;

class obj_info
{
private:
    int cls_id;
    int left;
    int top; 
    int width;
    int height;

public:
    obj_info(int _id=0, int _l=0, int _t=0, int _w=0, int _h=0):
        cls_id(_id), left(_l), top(_t), width(_w), height(_h) {}
};

int frame_num = 0;
std::vector<obj_info> infos;
// File path for the single File Source
static const std::wstring file_path(L"/opt/nvidia/deepstream/deepstream/samples/streams/sample_qHD.mp4");
std::ofstream f("pred.txt");

//
// Function to be called on XWindow KeyRelease event
//

void xwindow_key_event_handler(const wchar_t* in_key, void* client_data)
{   
    std::wstring wkey(in_key); 
    std::string key(wkey.begin(), wkey.end());
    std::cout << "key released = " << key << std::endl;
    key = std::toupper(key[0]);
    if(key == "P"){
        dsl_pipeline_pause(L"pipeline");
    } else if (key == "R"){
        dsl_pipeline_play(L"pipeline");   
    } else if (key == "Q"){
        dsl_pipeline_stop(L"pipeline");
        dsl_main_loop_quit();
    }
}

class ReportData {
public:
    int m_report_count;
    int m_header_interval;

    ReportData(int count, int interval) : m_report_count(count), m_header_interval(interval) {}
};

boolean dsl_pph_meter_cb(double* session_fps_averages, double* interval_fps_averages, 
    uint source_count, void* client_data)
{
    // cast the C void* client_data back to a py_object pointer and deref
    // report_data = cast(client_data, POINTER(py_object)).contents.value
    ReportData *report_data = static_cast<ReportData*>(client_data);

    // Print header on interval
    if (report_data->m_report_count % report_data->m_header_interval == 0) {
        std::wstring header = L"";

        for(int i=0; i<source_count; i++) {  
            header += L"FPS ";
            header += std::to_wstring(i);
            header += L" (AVG)";
        }
        std::wcout << header << "\n";
    }

    // Print FPS counters
    std::wstring counters = L"";

    for(int i=0; i<source_count; i++) {
        counters += std::to_wstring(interval_fps_averages[i]);
        counters += L" ";
        counters += std::to_wstring(session_fps_averages[i]);
    }

    std::wcout << counters << "\n";

    // Increment reporting count
    report_data->m_report_count += 1;

    return true;
}

void xwindow_delete_event_handler(void *client_data)
{
    std::cout<<"delete window event"<<std::endl;
    dsl_pipeline_stop(L"pipeline");
    dsl_main_loop_quit();
}

void eos_event_listener(void *client_data)
{
    std::cout<<"Pipeline EOS event"<<std::endl;
    dsl_pipeline_stop(L"pipeline");
    dsl_main_loop_quit();
}

// 
// Function to be called on every change of Pipeline state
// 
void state_change_listener(uint old_state, uint new_state, void* client_data)
{
    std::cout<<"previous state = " << dsl_state_value_to_string(old_state) 
        << ", new state = " << dsl_state_value_to_string(new_state) << std::endl;
}

void dsl_ode_monitor_occurrence(dsl_ode_occurrence_info* info, void* client_data)
{
    boolean frame_start = *(boolean *)client_data;
    if (frame_start == false) {
        frame_start = true;
        frame_num = info->source_info.frame_num;
    }
    else if (frame_num != info->source_info.frame_num){
        // shared memory write;

        frame_num = info->source_info.frame_num;
        infos.clear();
        infos.resize(100);
    }

    std::wcout << "Trigger Name:         " << info->trigger_name << "\n";
    std::cout << "Unique Id:             " << info->unique_ode_id << "\n";
    std::cout << "NTP Timestamp:         " << info->ntp_timestamp << "\n";
    std::cout << "Source Data:           ----------------------" << "\n";
    std::cout << "Id:                    " << info->source_info.source_id << "\n";
    std::cout << "Batch Id:              " << info->source_info.batch_id << "\n";
    std::cout << "Pad Index:             " << info->source_info.pad_index << "\n";
    std::cout << "Frame Num:             " << info->source_info.frame_num << "\n";
    std::cout << "Frame Width:           " << info->source_info.frame_width << "\n";
    std::cout << "Frame Height:          " << info->source_info.frame_height << "\n";
    std::cout << "Infer Done:            " << info->source_info.inference_done << "\n";
    std::cout << "Is Object Occurrence?: " << info->is_object_occurrence << "\n";

    if (info->is_object_occurrence) {
        std::cout << "Object Data:           ----------------------" << "\n";
        std::cout << "Class Id:                       " << info->object_info.class_id << "\n";
        std::cout << "Tracking Id:                    " << info->object_info.tracking_id << "\n";
        std::wcout << "Label:                         " << info->object_info.label << "\n";
        std::cout << "Left:                           " << info->object_info.left << "\n";
        std::cout << "Top:                            " << info->object_info.top << "\n";
        std::cout << "Width:                          " << info->object_info.width << "\n";
        std::cout << "Height:                         " << info->object_info.height << "\n";
        std::cout << "Inference confidence:           " << info->object_info.inference_confidence << "\n";
        std::cout << "Tracker confidence:             " << info->object_info.tracker_confidence << "\n";

        // <frame>, <id>, <bb_left>, <bb_top>, <bb_width>, <bb_height>, <conf>, <x>, <y>, <z>
        f << info->source_info.frame_num << ' ' << info->object_info.tracking_id << ' ' \
            << info->object_info.left << ' ' << info->object_info.top << ' ' \
            << info->object_info.width << ' ' << info->object_info.height << ' ' \
            << info->object_info.tracker_confidence << ' ' << -1 << ' ' << -1 << ' ' << -1 << '\n';

        infos.emplace_back(obj_info(info->object_info.class_id, info->object_info.left, info->object_info.top, info->object_info.width, info->object_info.height));
    }
}

int main(int argc, char** argv)
{
    DslReturnType retval;

    if (!f) {
        std::wcout << L"Unable to open file" << "\n";
        return -1;
    }

    while (true) {  
        ReportData report_data(0, 12);
        boolean frame_start = false;
        infos.clear();
        infos.resize(100);

        retval = dsl_ode_trigger_occurrence_new(L"every-occurrence", DSL_ODE_ANY_SOURCE, DSL_ODE_ANY_CLASS, DSL_ODE_TRIGGER_LIMIT_NONE);
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_ode_trigger_occurrence_new: " << dsl_return_value_to_string(retval) << "\n";
            return retval;
        }

        int frame_number =0 ;

        // Create the forground and background colors for our custom, formated Object Label 
        retval = dsl_display_type_rgba_color_custom_new(L"full-white", 
            1.0, 1.0, 1.0, 1.0);
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_display_type_rgba_color_custom_new: " << dsl_return_value_to_string(retval) << "\n";
            return retval;
        }
        retval = dsl_display_type_rgba_color_custom_new(L"opaque-black", 
            0.0, 0.0, 0.0, 0.8);
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_display_type_rgba_color_custom_new: " << dsl_return_value_to_string(retval) << "\n";
            return retval;
        }

        retval = dsl_display_type_rgba_font_new(L"verdana-bold-16-white", 
            L"verdana bold", 16, L"full-white");
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_display_type_rgba_font_new: " << dsl_return_value_to_string(retval) << "\n";
            return retval;
        }

        retval = dsl_ode_action_format_label_new(L"format-label", 
            L"verdana-bold-16-white", 
            true, 
            L"opaque-black");
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_ode_action_format_label_new: " << dsl_return_value_to_string(retval) << "\n";
            return retval;
        }

        retval = dsl_display_type_rgba_color_palette_random_new(L"random-color", 80, DSL_COLOR_HUE_RANDOM, DSL_COLOR_LUMINOSITY_RANDOM, 1.0, 1000);
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_display_type_rgba_color_custom_new: " << dsl_return_value_to_string(retval) << "\n";
            return retval;
        }

        retval = dsl_ode_action_format_bbox_new(L"format-bbox", 5, L"random-color", false, nullptr);
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_ode_action_format_bbox_new: " << dsl_return_value_to_string(retval) << "\n";
            return retval;
        }  

        retval = dsl_ode_action_monitor_new(L"monitor-action", dsl_ode_monitor_occurrence, &frame_start);
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_ode_action_monitor_new: " << dsl_return_value_to_string(retval) << "\n";
            return retval;
        }  

        const wchar_t* label_bbox_components[] = { L"format-bbox", L"format-label", L"monitor-action", nullptr};
        retval = dsl_ode_trigger_action_add_many(L"every-occurrence", label_bbox_components);
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_ode_trigger_action_add_many: " << dsl_return_value_to_string(retval) << "\n";
            return retval;
        }  

        retval = dsl_pph_ode_new(L"ode-handler");
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_pph_ode_new: " << dsl_return_value_to_string(retval) << "\n";
            return retval;
        }  

        retval = dsl_pph_ode_trigger_add(L"ode-handler", L"every-occurrence");
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_pph_ode_trigger_add: " << dsl_return_value_to_string(retval) << "\n";
            return retval;
        }  

        retval = dsl_pph_meter_new(L"meter-pph", 1, dsl_pph_meter_cb, &report_data);
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_pph_meter_new: " << dsl_return_value_to_string(retval) << "\n";
            return retval;
        } 

        // # For each camera, create a new RTSP Source for the specific RTSP URI
        dsl_source_file_new(L"file-source", file_path.c_str(), false);
        if (retval != DSL_RESULT_SUCCESS) {
            std::cout << "error, dsl_source_file_new: " << dsl_return_value_to_string(retval) << "\n";
            return retval;
        }  

        retval = dsl_infer_gie_primary_new(L"primary-gie", primary_infer_config_file.c_str(), primary_model_engine_file.c_str(), 2);
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_infer_gie_primary_new: " <<  dsl_return_value_to_string(retval) << std::endl;
            break;
        }

        retval = dsl_tracker_dcf_new(L"dcf-tracker", tracker_config_file.c_str(), 640, 384, true, false);
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_tracker_dcf_new: " <<  dsl_return_value_to_string(retval) << std::endl;
            break;
        }

        retval = dsl_osd_new(L"on-screen-display", true, true, true, false);
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_osd_new: " <<  dsl_return_value_to_string(retval) << std::endl;
            break;
        }

        retval = dsl_osd_pph_add(L"on-screen-display", L"ode-handler", DSL_PAD_SINK);
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, on-screen-display: " << dsl_return_value_to_string(retval) << "\n";
            return retval;
        } 

        retval = dsl_osd_pph_add(L"on-screen-display", L"meter-pph", DSL_PAD_SINK);
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, on-screen-display: " << dsl_return_value_to_string(retval) << "\n";
            return retval;
        }

        // # New Window Sink, 0 x/y offsets and dimensions 
        retval = dsl_sink_window_new(L"window-sink", 0, 0, sink_width, sink_height);
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, window-sink: " <<  dsl_return_value_to_string(retval) << std::endl;
            break;
        }

        // # Add all the components to a new pipeline
        const wchar_t* components[] = { L"file-source",L"primary-gie",
            L"dcf-tracker", L"on-screen-display",L"window-sink",nullptr};
        retval = dsl_pipeline_new_component_add_many(L"pipeline", components);            
        if (retval != DSL_RESULT_SUCCESS) {
            std::wcout << "error, dsl_pipeline_new_component_add_many: " <<  dsl_return_value_to_string(retval) << std::endl;
            break;
        }

        // # Add the XWindow event handler functions defined above
        retval = dsl_pipeline_xwindow_key_event_handler_add(L"pipeline", 
            xwindow_key_event_handler, nullptr);
        if (retval != DSL_RESULT_SUCCESS) break;
        retval = dsl_pipeline_xwindow_delete_event_handler_add(L"pipeline", 
            xwindow_delete_event_handler, nullptr);
        if (retval != DSL_RESULT_SUCCESS) break;

        // # Add the listener callback functions defined above
        retval = dsl_pipeline_state_change_listener_add(L"pipeline", 
            state_change_listener, nullptr);
        if (retval != DSL_RESULT_SUCCESS) break;
        retval = dsl_pipeline_eos_listener_add(L"pipeline", 
            eos_event_listener, nullptr);
        if (retval != DSL_RESULT_SUCCESS) break;

        // # Play the pipeline
        retval = dsl_pipeline_play(L"pipeline");
        if (retval != DSL_RESULT_SUCCESS) break;

        // # Join with main loop until released - blocking call
        dsl_main_loop_run();

        f.close();
        retval = DSL_RESULT_SUCCESS;
        break;
    }

    // # Print out the final result
    std::wcout << dsl_return_value_to_string(retval) << std::endl;

    dsl_pipeline_delete_all();
    dsl_component_delete_all();

    std::cout<<"Goodbye!"<<std::endl;  
    return 0;
}

[rjhowell44]

@youngjae-avikus if the MOTChallenge-format is this important, I think we should consider adding the support directly to DSL.

One thought would be to update the ODE File Action so that it supports three formats; TEXT, CSV, and MOTC . Or create a new action specifically for this.

[rjhowell44]

Will be added to DSL directly with #799