Sir,I try to run your code.But i have not clear idea about which files needs to be added.It means which header and source files required.could you please tell me how to run this project correctly.

Hi! Do you work with Windows and MS VC? In any case you need a CMake and configure project with it.

Dear Sir,I worked with visual studio 2010 and opencv 2.4.9. Sir,other problem is,all the header files and source files should be added or not,i want to tracking moving people video

Sir,After install Cmake then waht to do

can you step by step how to compile project. I installed the CMake and also set the path.

I have bad news for you. This project requires OpenCV 3.x and compiler with C++11 standart support. Visual studio 2010 and opencv 2.4.9 are not supported - too old versions.

Dear Sir,I installd visual studio 2015 and opencv 3.1.Then what shall i do sir,explain me sir,I am a beginner for this subject.Your kindly support i will hope

On Tuesday, August 22, 2017 1:28 PM, Sergey Nuzhny <notifications@github.com> wrote:

I have bad news for you. This project requires OpenCV 3.x and compiler with C++11 standart support. Visual studio 2010 and opencv 2.4.9 are not supported - too old versions.— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

Sir,I have a code of tracking i attached that file.it worked only to MOG2,could you please look at that code and tell me what modification needs to be required.

On Wednesday, August 23, 2017 6:54 AM, Dulitha Sadhanuwan <dulithasadhanuwan@yahoo.com> wrote:

Dear Sir,I installd visual studio 2015 and opencv 3.1.Then what shall i do sir,explain me sir,I am a beginner for this subject.Your kindly support i will hope

On Tuesday, August 22, 2017 1:28 PM, Sergey Nuzhny <notifications@github.com> wrote:

include "opencv2/imgcodecs.hpp"

include "opencv2/imgproc.hpp"

include "opencv2/videoio.hpp"

include <opencv2/highgui.hpp>

include <opencv2/video.hpp>

include

include "TrackedObject.h"

using namespace cv; using namespace std;

// Global constant

const bool SHOW_ALL_OPERATION_RESULTS = true; // defines if different operation results should be shown const bool WINDOW_NAMES_READABLE = false; // if set, creates y-offset to make window names readable const int ELEMENTS_PER_ROW = 3; // defines how much windows are shown within a row const bool TRACKING_RESULTS_IN_ORIGINAL = false; // shows tracking results in original frame

// video related constants const int VIDEO_NUMBER = 1; // 1:10, choose different number for different video const bool LOOP_VIDEO = true; // defines if video should be looped infinitely

const char* VIDEO_DIRECTORY_PATH = ".\detect.avi\"; // define path to video directory

const bool SOURCE2GRAY = false; // defines, if source frame should be converted to gray

// foreground segmentation postprocessing related constants const int HISTORY = 1000; // MOG2 parameter: amount of considered frames const double VAR_THRESHOLD = 20; // MOG2 parameter: threshold for degree of minimum variance const bool DETECT_SHADOWS = false; // MOG2 parameter: defines if shadows should be detected

int thresholdMask = 100; // defines start value of threshold for // binarization of foreground frame const int THRESHOLD_MASK_MAX = 255; // defines maximum value for threshold

// morphing related constants const int KERNEL_TYPE = MORPH_ELLIPSE; // choose: MORPH_RECT, MORPH_CROSS, MORPH_ELLIPSE int kernelSizeClose = 5; // defines start value for kernel size of close op. int kernelSizeErode = 5; // defines start value for kernel size of erode op. int kernelSizeDilate = 5; // defines start value for kernel size of dilate op. const int KERNEL_SIZE_MAX = 20; // defines maximum value for threshold

// drawing related constants const Scalar DEFAULT_DRAWING_COLOR = Scalar(0, 255, 0); // defines default color of bounding boxes and labels const int CONTOUR_MIN_AREA = 500; // defines minimum area of contours const int BBOX_MIN_AREA = 3000; // defines minimum area of bounding boxes const bool SHOW_LABELS = false; // shows labels of object adjusted to the bounding box const int HEATMAP_RANGE = 25; // defines size of different colors in heat map // and threshold (color of highest weight)

// bounding box and label related constants const float ALPHA = 0.5; // defines opacity of rectangle and label const int LABEL_HEIGHT = 20; // defines height of label const char* TEST_NAME = "Car"; // defines name for test labels

const int TEXT_FONT_FACE = 2; // defines font of label text const int TEXT_THICKNESS = 2; // defines thickness of label text const int TEXT_LINE_TYPE = 0; // defines line type of label text const double TEXT_FONT_SCALE = 0.7; // defines scale of label text

// tracking related constants const bool TRACKING = true; // defines if tracking is active const bool STATEFUL_TRACKING = true; // defines if stateful tracking with TrackedObjects is active const int MAX_COUNT_TOTAL = 150; // defines max no. of tracked points in total const int MAX_COUNT_PER_CAR = 25; // defines max no. of tracked points per car const int MIN_POINTS_TRACKED_OBJECT = 3; // defines min no. of points for tracked objects const int INIT_INTERVAL = 15; // defines, after how many frames the tracker should initialize new points const TermCriteria termcrit(TermCriteria::COUNT | TermCriteria::EPS, 20, 0.03); // defines criteria for klt tracker const Size subPixWinSize(10, 10); const Size winSize(31, 31);

/// Global variables

// window related variables int scale = 10; // scales windows (and mats) after processing before displaying // (10 is equal to standard size, 20 to doubled size)

// video related variables int frameStart = 1; // defines frame to start (set in video chooser) int frameEnd = 0; // defines frame to end (set in video chooser)

// frame related variables

Mat frame; //current frame Mat fgMaskMOG2; //fg mask fg mask generated by MOG2 method Mat frameGray; // current frame in grayscale Mat frameGrayPrev; // previous frame in grayscale Mat frameThreshold; // frame with applied threshold (to elimante shadows etc.) Mat frameMorphed; // morphed frame (to elimante noise) Mat frameMasked; // morphed frame applied as a mask to the original frame Mat frameContours; // drawn contours Mat frameKltTracking; // klt tracking lanes

Ptr pMOG2; //MOG2 Background subtractor

// tracking related variables bool resetFeatures = false; Point2f point; vector points[2]; int amountHandledContoursPrev = 0;

vector trackedObjects; int objectCounter;

int keyboard; //input from keyboard

/// Function Headers

char* videoChooser();

void processVideo(char* videoFilename);

//void threshold_demo(int,void*);

void showFrame(char* windowName, Mat frame, int width,int height, int position);

void resizeFrames();

void drawBoundingBoxes(vector<vector> contours, Mat frame);

void showControlPanel();

void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector> contours);

void trackObjects(vector<vector> contours, size_t currentPoint);

size_t* getCorrespondingObject(vector contour);

void handleProcessedPoints();

Scalar getHeatMapColor(float value);

void processVideo(char videoFilename); void processImages(char firstFrameFilename);

char* VIDEO_FILE = "visiontraffic.avi";

int main(int argc, char* argv[]) {

char*   videoPath = new char[100];

/// initialize variables
// define path of video file
strcpy(videoPath, VIDEO_DIRECTORY_PATH);
strcat(videoPath, videoChooser());

//create GUI windows
namedWindow("Original");
namedWindow("Foreground (MOG2)");
namedWindow("Threshold");
namedWindow("Morphed");
namedWindow("Contours");
namedWindow("Masked Original Frame");
namedWindow("KLT Tracking");

//create Background Subtractor objects
pMOG2 = createBackgroundSubtractorMOG2(HISTORY, VAR_THRESHOLD, DETECT_SHADOWS); //MOG2 approach

//input data coming from a video
processVideo(VIDEO_FILE);
//destroy GUI windows
destroyAllWindows();
return EXIT_SUCCESS;

}

void processVideo(char* videoFilename) {

/// variable declaration and initialization
// morphing related variables (initialized during each loop, since parameters are changable)
Mat kernelClose;
Mat kernelErode;
Mat kernelDilate;

// frame related variables
stringstream    frameNumberTotal;
stringstream    frameNumberCurrent;
stringstream    frameInfo;
int             frameWidth = 0,
                frameHeight = 0;

// contour related variables
vector<vector<Point>>   contours;
vector<Vec4i>           hierarchy;

// displaying related variables
int     r = 1;      // counter for row

//// video processing

//create the capture object
VideoCapture capture(videoFilename);
if (!capture.isOpened()) {
    //error in opening the video input
    cerr << "Unable to open video file: " << videoFilename << endl;
    exit(EXIT_FAILURE);
}

frameNumberTotal << capture.get(CAP_PROP_FRAME_COUNT);
frameWidth = (int)capture.get(CV_CAP_PROP_FRAME_WIDTH);
frameHeight = (int)capture.get(CV_CAP_PROP_FRAME_HEIGHT);

if (frameEnd == 0)
    frameEnd = (int)capture.get(CAP_PROP_FRAME_COUNT);

// set video to defined start frame
capture.set(CAP_PROP_POS_FRAMES, frameStart);

// show control panel (enables modification of some variables in realtime)
showControlPanel();

while ((char)keyboard != 27) {
    // read the current frame
    if (!capture.read(frame)) {
        cerr << "Unable to read next frame." << endl;
        cerr << "Exiting..." << endl;
        exit(EXIT_FAILURE);
    }

    // convert original frame to grayscale if specified by constant
    if (SOURCE2GRAY)
        frameGray.copyTo(frame);

    // re-initialize objects with changable parameters
    if (kernelSizeClose > 0)
        kernelClose = getStructuringElement(KERNEL_TYPE, Size(kernelSizeClose, kernelSizeClose));
    if (kernelSizeErode > 0)
        kernelErode = getStructuringElement(KERNEL_TYPE, Size(kernelSizeErode, kernelSizeErode));
    if (kernelSizeDilate > 0)
        kernelDilate = getStructuringElement(KERNEL_TYPE, Size(kernelSizeDilate, kernelSizeDilate));

    pMOG2->apply(frame, fgMaskMOG2);

///read input data. ESC or 'q' for quitting
    while ((char)keyboard != 'q' && (char)keyboard != 27) {
        //read the current frame
        if (!capture.read(frame)) {
            cerr << "Unable to read next frame." << endl;
            cerr << "Exiting..." << endl;
            exit(EXIT_FAILURE);
        }

        //update the background model
        pMOG2->apply(frame, fgMaskMOG2);

        stringstream ss;
        rectangle(frame, cv::Point(10, 2), cv::Point(100, 20),
            cv::Scalar(255, 255, 255), -1);
        ss << capture.get(CAP_PROP_POS_FRAMES);
        string frameNumberString = ss.str();
        putText(frame, frameNumberString.c_str(), cv::Point(15, 15),
            FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0, 0));

        imshow("Frame", frame);
        imshow("FG Mask MOG 2", fgMaskMOG2);

        //get the input from the keyboard
        keyboard = waitKey(30);

        //get the frame number and write it on the current frame
        rectangle(frame, Point(10, 2), Point(100, 20), Scalar(255, 255, 255), -1);
        frameNumberCurrent << capture.get(CAP_PROP_POS_FRAMES);
        frameInfo << frameNumberCurrent.str() << "/" << frameNumberTotal.str();
        putText(frame, frameInfo.str(), Point(15, 15),
            FONT_HERSHEY_SIMPLEX, 0.5, Scalar(0, 0, 0));

    }
        // apply threshold to MOG2 frame to get rid of shadows etc
        threshold(fgMaskMOG2, frameThreshold, thresholdMask, 255, THRESH_BINARY);

        // apply morphology operations to thresholded frame

        if (kernelSizeClose > 0)
            morphologyEx(frameMorphed, frameMorphed, MORPH_CLOSE, kernelClose);
        if (kernelSizeErode > 0)
            morphologyEx(frameMorphed, frameMorphed, MORPH_ERODE, kernelErode);
        if (kernelSizeDilate > 0)
            morphologyEx(frameMorphed, frameMorphed, MORPH_DILATE, kernelDilate);

        frameThreshold.copyTo(frameMorphed);

        // save previous masked frame as grayscale frame for later tracking
        if (!frameMasked.empty())
            cvtColor(frameMasked, frameGrayPrev, CV_RGB2GRAY);

        // apply morphed mask, but save before the morphed frame before
        frameMorphed.copyTo(frameContours);
        frameMorphed.copyTo(frameMasked);
        frame.copyTo(frameMasked, frameMasked);

        // save new masked frame as grayscale frame for later tracking, also save mask to KltTracking frame
        cvtColor(frameMasked, frameGray, CV_RGB2GRAY);
        frameMasked.copyTo(frameKltTracking);

        // draw and fill contours
        findContours(frameContours, contours, hierarchy, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
        setIdentity(frameContours, Scalar(0, 0, 0));
        drawContours(frameContours, contours, -1, Scalar(255, 255, 255), -1);

        // draws rectangle around identified contours 
        drawBoundingBoxes(contours, frameMasked);

        // prepare for and apply tracking algorithm (use grayscale copies of masked frame)
        if (!frameGrayPrev.empty() && TRACKING)
            applyKltTracking(frameGray, frameGrayPrev, frameKltTracking, contours);

        /// show the different frames (from last to first row, to overlay windows bar)
        resizeFrames();             // resize all frames

    if (SHOW_ALL_OPERATION_RESULTS) {
        // row 3
        r = 3;
        showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);

        // row 2
        r = 2;
        showFrame("Morphed", frameMorphed, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
        showFrame("Contours", frameContours, frameWidth, frameHeight, 2 + (r - 1)*ELEMENTS_PER_ROW);
        showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 3 + (r - 1)*ELEMENTS_PER_ROW);

        // row 1
        r = 1;
        showFrame("Original", frame, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
        showFrame("Foreground (MOG2)", fgMaskMOG2, frameWidth, frameHeight, 2 + (r - 1)*ELEMENTS_PER_ROW);
        showFrame("Threshold", frameThreshold, frameWidth, frameHeight, 3 + (r - 1)*ELEMENTS_PER_ROW);

    }
    else {
        // row 1
        r = 1;
        showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
        showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 2 + (r - 1)*ELEMENTS_PER_ROW);
    }

    // shows image for the specified amount of milliseconds 
    keyboard = waitKey(30);

    switch (keyboard) {
    case 'r':
        resetFeatures = true;       // reset on pressed 'r'
        break;
    }

    // loop through the video if end is reached and loop is set
    if (LOOP_VIDEO && (frameNumberCurrent.str().compare(std::to_string(frameEnd)) == 0))
        capture.set(CAP_PROP_POS_FRAMES, frameStart);

    // ensure that new feature points are initialized every INIT_INTERVAL frames for the tracker
    if (std::stoi(frameNumberCurrent.str()) % INIT_INTERVAL == 0 && trackedObjects.size() > 0)
        resetFeatures = true;

    // cleaning operation: clean string streams
    frameNumberCurrent.str("");
    frameInfo.str("");
}

// cleaning operation: release capture object
capture.release();

    //imshow("FG Mask MOG 2", fgMaskMOG2);

    //get the input from the keyboard
    //keyboard = waitKey(30);
//}
//delete capture object
////capture.release();

}

char* videoChooser() {

// variable declaration
char*   videoFile = "dog.avi";  // saves name of selected video file

// chooses video file depending on chosen video number
switch (VIDEO_NUMBER) {
case 1:
    videoFile = "visiontraffic.avi";
    frameStart = 90;
    frameEnd = 440;
    break;

case 2:
    videoFile = "viptraffic.avi";
    break;

case 3:
    videoFile = "highwayII_raw.avi";
    break;

case 4:
    videoFile = "dispersed.ogv";
    break;

case 5:
    videoFile = "highspeed.ogv";
    break;

case 6:
    videoFile = "hightraffic-dark.ogv";
    break;

case 7:
    videoFile = "hightraffic-occlude.ogv";
    break;

case 8:
    videoFile = "threecars.ogv";
    break;

case 9:
    videoFile = "turntraffic-dark.ogv";
    break;

case 10:
    videoFile = "visor_1261565143617_cesta1.wmv";
    break;

default:
    break;
}

return videoFile;

}

void showFrame(char* windowName, Mat frame, int width, int height, int position) { // variable declaration and initialization int x = 0, // x-coordinates for position y = 0, // y-coordinates for position yOffset = 30; // offset for y-coordinates, since windows bar does not count into frame height // strategy for y-offset: create windows from last row to first to overlay the bar

// create y-Offset if defince to make titles of windows readable
if (!WINDOW_NAMES_READABLE) {
    yOffset += -32;
}
else if (position > ELEMENTS_PER_ROW) {
    yOffset += 32;
}

// create window and display defined frame

cv::namedWindow(windowName);
cv::imshow(windowName, frame);

// resize stats for position calculation, if defined
if (scale != 10) {
    // filter for the case, that it's zero, since range is not adjustable in opencv gui
    if (scale == 0)
        scale = 1;

    // calculate scalefactor from value of scale variable
    float scaleFactor = (float)scale / 10;

    // calculate new width and height
    width = (int)(width * scaleFactor);
    height = (int)(height * scaleFactor);
}

//// get positioning

// get x coordinates with width
if (((position - 1) % ELEMENTS_PER_ROW) != 0)
    x = ((position - 1) % ELEMENTS_PER_ROW) * width;

// get y coordinates with height
if (position > ELEMENTS_PER_ROW)
    y = (position - (position - 1) % ELEMENTS_PER_ROW) / ELEMENTS_PER_ROW * height;

// place window at calculated position
moveWindow(windowName, x, y + yOffset);

}

void resizeFrames() { //resize all frames, if defined if (scale != 10 && scale > 0) { // filter for the case, that it's zero, since range is not adjustable in opencv gui if (scale == 0) scale = 1;

    // calculate scalefactor from value of scale variable
    float scaleFactor = (float)scale / 10;

    // resize all frames
    resize(frame, frame, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
    resize(fgMaskMOG2, fgMaskMOG2, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
    resize(frameThreshold, frameThreshold, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
    resize(frameMorphed, frameMorphed, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
    resize(frameMasked, frameMasked, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
    resize(frameContours, frameContours, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
    resize(frameKltTracking, frameKltTracking, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
}

}

void drawBoundingBoxes(vector<vector> contours, Mat frame) { // variable declaration int amountHandledContours = 0; Scalar drawingColor = DEFAULT_DRAWING_COLOR;

// create bounding rect of object and draw on defined frame
for (vector<vector<Point>>::iterator itc = contours.begin(); itc != contours.end(); itc++) {
    // variable declaration and initialization
    Rect    boundingBox = boundingRect(Mat(*itc));
    Mat     overlay;

    // only handle rectangles with a minimum area
    if (boundingBox.area() >= CONTOUR_MIN_AREA) {
        if (contourArea(*itc) >= CONTOUR_MIN_AREA) {

            // get corresponding object, if there's one, to define the drawing color
            size_t* indexCorrespondingObject = getCorrespondingObject(*itc);
            if (indexCorrespondingObject != NULL)
                drawingColor = trackedObjects[*indexCorrespondingObject].getColor();

            // draw contours regardless of area
            drawContours(frame, contours, -1, drawingColor, 1);

            // handle bounding boxes (but only if they're bigger than defined minimum area)
            if (boundingBox.area() >= BBOX_MIN_AREA) {
                // copy frame to
                frame.copyTo(overlay);

                // draw rectangle 
                rectangle(overlay, boundingBox, drawingColor, 3);

                // draw test label if constant is set
                if (SHOW_LABELS)
                    rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y),
                    Point(boundingBox.x + boundingBox.width + 1,
                    boundingBox.y - LABEL_HEIGHT), drawingColor, -1);

                // combine transparent overlay of bounding rect with original frame
                cv::addWeighted(overlay, ALPHA, frame, 1 - ALPHA, 0, frame);

                // afterwards, put text with object name into the label, if constant is set
                if (SHOW_LABELS)
                    putText(frame, TEST_NAME, Point(boundingBox.x + 10, boundingBox.y - 2),
                    TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(0, 0, 0),
                    TEXT_THICKNESS, TEXT_LINE_TYPE, false);

                // count handled contours
                amountHandledContours++;
            }
        }
    }
}

// re-initialize feature tracker if new objects appeared in this frame
if (amountHandledContours > amountHandledContoursPrev) {
    resetFeatures = true;
    cout << amountHandledContours << "/" << amountHandledContoursPrev << endl;
}

amountHandledContoursPrev = amountHandledContours;

}

void showControlPanel() { // constant declaration const int WIDTH = 400, HEIGHT = 240;

const char* nameWindow = "Control Panel";
const char* nameTrackbarWindowScale = "Scale";
const char* nameTrackbarThreshold = "Threshold";
const char* nameTrackbarKernelSizeClose = "SizeClose";
const char* nameTrackbarKernelSizeErode = "SizeErode";
const char* nameTrackbarKernelSizeDilate = "SizeDilate";

// create window
namedWindow(nameWindow, CV_WINDOW_AUTOSIZE | CV_GUI_NORMAL);
resizeWindow(nameWindow, WIDTH, HEIGHT);

// window relating parameters
createTrackbar(nameTrackbarWindowScale, nameWindow, &scale, 20, NULL);
createTrackbar(nameTrackbarThreshold, nameWindow, &thresholdMask, THRESHOLD_MASK_MAX, NULL);

createTrackbar(nameTrackbarKernelSizeClose, nameWindow, &kernelSizeClose, KERNEL_SIZE_MAX, NULL);
createTrackbar(nameTrackbarKernelSizeErode, nameWindow, &kernelSizeErode, KERNEL_SIZE_MAX, NULL);
createTrackbar(nameTrackbarKernelSizeDilate, nameWindow, &kernelSizeDilate, KERNEL_SIZE_MAX, NULL);

}

/**

@function for klt tracking */ void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector> contours) {

/// variable declaration

vector newPoints;

/// calculate features and then the optical flow if (resetFeatures) { // automatic initialization goodFeaturesToTrack(frameGray, newPoints, MAX_COUNT_TOTAL, 0.01, 10, Mat(), 3, 0, 0.04); cornerSubPix(frameGray, newPoints, subPixWinSize, Size(-1, -1), termcrit);

//points[0].clear();    // could solve performance issue, but creates strange behaviour: 
// trackedObjects also reset every reset
//points[1].clear();

// initialize with all available points from tracked objects
for (std::vector<TrackedObject>::size_type l = 0; l != trackedObjects.size(); l++)
    for (std::vector<TrackedObject>::size_type j = 0; j != trackedObjects[l].getPoints().size(); j++) {
        points[0].push_back(trackedObjects[l].getPoints()[j]);
    }

} if (!points[0].empty() || trackedObjects.size() > 0) { // variable declaration vector status; vector err; size_t i, k;

// check if previous frame is existing, otherwise copy the current one
if (frameGrayPrev.empty())
    frameGray.copyTo(frameGrayPrev);

// calculate the optical flow with KLT
calcOpticalFlowPyrLK(frameGrayPrev, frameGray, points[0], points[1], status, err, winSize,
    3, termcrit, 0, 0.001);

/// iterate through all found points
for (i = k = 0; i < points[1].size(); i++) {

    // don't handle point, if there's no corresponding point from the last frame
    if (!status[i])
        continue;

    // otherwise save and draw point (shrink's point vector, resizes after the for loop)
    points[1][k++] = points[1][i];

    /// get corresponding contour and object, otherwise create new tracked object
    if (STATEFUL_TRACKING)
        trackObjects(contours, i);
    else
        circle(frameKltTracking, points[1][i], 6, DEFAULT_DRAWING_COLOR, 2, 8);
}

points[1].resize(k);

}

// handle the processed points handleProcessedPoints();

// save that initialization was done and not needed anymore, also add new points if (resetFeatures) { resetFeatures = false;

for (std::vector<Point2f>::size_type l = 0; l != newPoints.size(); l++)
    points[1].push_back(newPoints[l]);

}

// save the current values as the previous ones if (!resetFeatures) std::swap(points[1], points[0]);

cv::swap(frameGrayPrev, frameGrayPrev); }

void trackObjects(vector<vector> contours, size_t currentPoint) { /// variable declaration int containedInContour; bool foundObject = false; bool foundContour = false; bool foundObjectOfContour = false; vector::size_type m; // iterator for contours size_t* correspondingObject = NULL; // pointer for corresponding object

/// get contour of point
for (m = 0; m != contours.size(); m++) {
    // checks if point is inside or on the edge of the contour
    containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false);

    // exit loop if corresponding contour is found
    if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) {
        foundContour = true;
        break;
    }
}

if (trackedObjects.size() > 0) {
    // check for each point in the tracked object if it already exists
    for (std::vector<TrackedObject>::size_type l = 0; l != trackedObjects.size(); l++) {
        size_t* containedInObject = NULL;

        if (trackedObjects[l].containsOldPoint(points[0][currentPoint]) != NULL) {
            // add the point (it will increase weight automatically instead)
            trackedObjects[l].addPoint(points[1][currentPoint], points[0][currentPoint]);
            foundObject = true;

            // update contour of object, if contour was not modified this frame yet
            bool foundContour = false;
            if (!trackedObjects[l].getContourStatus()) {
                std::vector<Point>::size_type m;
                for (m = 0; m != contours.size(); m++) {
                    // checks if point is inside or on the edge of the contour
                    containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false);

                    // exit loop if corresponding contour is found
                    if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) {
                        foundContour = true;
                        break;
                    }
                }

                // update corresponding contour, if one was found
                if (foundContour)
                    trackedObjects[l].setContour(contours[m]);
            }
        }

    }
}

// create new object with this point
if (!foundObject && foundContour) {
    size_t* containedInObject;
    containedInObject = getCorrespondingObject(contours[m]);

    if (containedInObject != NULL) {
        // contour belongs to an existing object, so add the point
        size_t containedInObjectVal = *containedInObject;

        // add points 
        if (trackedObjects[containedInObjectVal].getPoints().size() < MAX_COUNT_PER_CAR)
            trackedObjects[containedInObjectVal].addPoint(points[1][currentPoint], points[0][currentPoint]);

        foundObjectOfContour = true;
    }

    // create new tracked object, since there's no existing one with this contour
    if (!foundObjectOfContour) {
        char name[11] = "Object_";
        char objectNumber[4];
        sprintf(objectNumber, "%d", (int)(objectCounter + 1));
        strcat(name, objectNumber);

        TrackedObject* trObj = new TrackedObject(name);
        trObj->addPoint(points[1][currentPoint], points[0][currentPoint]);

        trObj->setContour(contours[m]);

        trackedObjects.push_back(*trObj);
        objectCounter++;
    }
}

}

size_t getCorrespondingObject(vector contour) { // variable declaration size_t position = NULL;

// find out if contour is assigned to existing object
if (trackedObjects.size() > 0 && contour.size() > 0) {
    for (std::vector<TrackedObject>::size_type i = 0; i != trackedObjects.size(); i++) {
        if (trackedObjects[i].getContour().size() == contour.size()) {
            double comparisonValue;
            comparisonValue = matchShapes(trackedObjects[i].getContour(), contour, CV_CONTOURS_MATCH_I1, NULL);
            if (comparisonValue == 0) {
                position = &i;
                break;
            }
        }
    }
}

return position;

}

void handleProcessedPoints() { // get original frame, if constant defines, that results should be shown in original if (TRACKING_RESULTS_IN_ORIGINAL) frame.copyTo(frameKltTracking);

if (trackedObjects.size() > 0) {
    for (std::vector<TrackedObject>::size_type i = 0; i != trackedObjects.size(); i++) {

        // process all points according to the status
        trackedObjects[i].processStatus();

        if (trackedObjects[i].getPoints().size() <= MIN_POINTS_TRACKED_OBJECT) {
            trackedObjects.erase(trackedObjects.begin() + i);
            i--;
            continue;
        }

        /// draw bounding box 
        Rect    boundingBox = boundingRect(trackedObjects[i].getContour());
        Mat overlay;
        frameKltTracking.copyTo(overlay);

        /// draw rectangle and label
        rectangle(overlay, boundingBox, trackedObjects[i].getColor(), 3);
        rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y),
            Point(boundingBox.x + boundingBox.width + 1,
            boundingBox.y - LABEL_HEIGHT), trackedObjects[i].getColor(), -1);

        // combine transparent overlay of bounding rect with original frame
        cv::addWeighted(overlay, ALPHA, frameKltTracking, 1 - ALPHA, 0, frameKltTracking);

        // afterwards, put text with object name into the label, if constant is set
        putText(frameKltTracking, trackedObjects[i].getName(), Point(boundingBox.x + 10, boundingBox.y - 2),
            TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(200, 200, 200),
            TEXT_THICKNESS, TEXT_LINE_TYPE, false);

        /// draw all feature points in the color of the corresponding object
        for (std::vector<Point2f>::size_type j = 0; j != trackedObjects[i].getPoints().size(); j++) {
            // handle weight of point
            int w = trackedObjects[i].getWeight()[j];
            int size = (int)trackedObjects[i].getPoints().size();

            if (w != -1) {  // point not found
                // draw filled circle for new points
                if (w == 1)
                    w = -1;
                else if (w >= HEATMAP_RANGE)
                    w = HEATMAP_RANGE;

                if (w != -1)
                    circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, getHeatMapColor(1 - (float)w / (float)HEATMAP_RANGE), -1, 8);
                circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, trackedObjects[i].getColor(), 2, 8);
            }
        }
    }
}

}

Scalar getHeatMapColor(float value) { float red, green, blue; const int NUM_COLORS = 4; static float color[NUM_COLORS][3] = { { 0, 0, 1 }, { 0, 1, 0 }, { 1, 1, 0 }, { 1, 0, 0 } }; // A static array of 4 colors: (blue, green, yellow, red) using {r,g,b} for each.

int idx1;        // |-- Our desired color will be between these two indexes in "color".
int idx2;        // |
float fractBetween = 0;  // Fraction between "idx1" and "idx2" where our value is.

if (value <= 0) { idx1 = idx2 = 0; }    // accounts for an input <=0
else if (value >= 1) { idx1 = idx2 = NUM_COLORS - 1; }    // accounts for an input >=0
else
{
    value = value * (NUM_COLORS - 1);       
    idx1 = (int)floor(value);               // Our desired color will be after this index.
    idx2 = idx1 + 1;                        // ... and before this index (inclusive).
    fractBetween = value - float(idx1);     // Distance between the two indexes (0-1).
}

red = ((color[idx2][0] - color[idx1][0])*fractBetween + color[idx1][0]) * 255;
green = ((color[idx2][1] - color[idx1][1])*fractBetween + color[idx1][1]) * 255;
blue = ((color[idx2][2] - color[idx1][2])*fractBetween + color[idx1][2]) * 255;

return Scalar(red, green, blue);

}

Sir,I have a code of tracking i attached that file.it worked only to MOG2,could you please look at that code and tell me what modification needs to be required.

Hi! I don't know what do you want. How can I advise you? This project - Multitarget-tracker contains full stack of algorithms for motion detection. I can tell you how it works.

You need CMake gui application, start it.
Select folder Mulitarget-tracker (with CMakeLists.txt file) as source code directory.
Select subfolder build as.. build directory.
Push a button Configure, select your MSVC compiler. Wait.
If OpenCV wasn't founded automatic then select OpenCV_DIR manually. Push a button Configure again.
Push a button Generate.
Now the folder build contains a solution file (.sln). Lets open it in Visual studio! The end.

Thank you very much sir,sir i have code can you find what is bug in the code please,it works only to MOG2

On Wednesday, August 23, 2017 9:16 AM, Sergey Nuzhny <notifications@github.com> wrote:

Sir,I have a code of tracking i attached that file.it worked only to MOG2,could you please look at that code and tell me what modification needs to be required. Hi! I don't know what do you want. How can I advise you? This project - Multitarget-tracker contains full stack of algorithms for motion detection. I can tell you how it works.

You need CMake gui application, start it.
Select folder Mulitarget-tracker (with CMakeLists.txt file).
Select subfolder folder build.
Push a button Configure, select your MSVC compler. Wait.
If OpenCV wasn't founded automatic then select OpenCV_DIR manually. Push a button Configure again.
Push a button Generate.
Now the folder build contains a solution file (.sln). Lets open it in Visual studio!
The end. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

include "opencv2/imgcodecs.hpp"

include "opencv2/imgproc.hpp"

include "opencv2/videoio.hpp"

include <opencv2/highgui.hpp>

include <opencv2/video.hpp>

include

include "TrackedObject.h"

using namespace cv; using namespace std;

// Global constant

// video related constants const int VIDEO_NUMBER = 1; // 1:10, choose different number for different video const bool LOOP_VIDEO = true; // defines if video should be looped infinitely

const char* VIDEO_DIRECTORY_PATH = ".\detect.avi\"; // define path to video directory

const bool SOURCE2GRAY = false; // defines, if source frame should be converted to gray

int thresholdMask = 100; // defines start value of threshold for // binarization of foreground frame const int THRESHOLD_MASK_MAX = 255; // defines maximum value for threshold

/// Global variables

// window related variables int scale = 10; // scales windows (and mats) after processing before displaying // (10 is equal to standard size, 20 to doubled size)

// video related variables int frameStart = 1; // defines frame to start (set in video chooser) int frameEnd = 0; // defines frame to end (set in video chooser)

// frame related variables

Ptr pMOG2; //MOG2 Background subtractor

// tracking related variables bool resetFeatures = false; Point2f point; vector points[2]; int amountHandledContoursPrev = 0;

vector trackedObjects; int objectCounter;

int keyboard; //input from keyboard

/// Function Headers

char* videoChooser();

void processVideo(char* videoFilename);

void showFrame(char* windowName, Mat frame, int width,int height, int position);

void resizeFrames();

void drawBoundingBoxes(vector<vector> contours, Mat frame);

void showControlPanel();

void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector> contours);

void trackObjects(vector<vector> contours, size_t currentPoint);

size_t* getCorrespondingObject(vector contour);

void handleProcessedPoints();

Scalar getHeatMapColor(float value);

void processVideo(char videoFilename); void processImages(char firstFrameFilename);

char* VIDEO_FILE = "visiontraffic.avi";

int main(int argc, char* argv[]) {

char*   videoPath = new char[100];

/// initialize variables
// define path of video file
strcpy(videoPath, VIDEO_DIRECTORY_PATH);
strcat(videoPath, videoChooser());

//create GUI windows
namedWindow("Original");
namedWindow("Foreground (MOG2)");
namedWindow("Threshold");
namedWindow("Morphed");
namedWindow("Contours");
namedWindow("Masked Original Frame");
namedWindow("KLT Tracking");

//create Background Subtractor objects
pMOG2 = createBackgroundSubtractorMOG2(HISTORY, VAR_THRESHOLD, DETECT_SHADOWS); //MOG2 approach

//input data coming from a video
processVideo(VIDEO_FILE);
//destroy GUI windows
destroyAllWindows();
return EXIT_SUCCESS;

}

void processVideo(char* videoFilename) {

//create the capture object VideoCapture capture(videoFilename); if (!capture.isOpened()) { //error in opening the video input cerr << "Unable to open video file: " << videoFilename << endl; exit(EXIT_FAILURE); }

///read input data. ESC or 'q' for quitting while ((char)keyboard != 'q' && (char)keyboard != 27) { //read the current frame if (!capture.read(frame)) { cerr << "Unable to read next frame." << endl; cerr << "Exiting..." << endl; exit(EXIT_FAILURE); }

//update the background model pMOG2->apply(frame, fgMaskMOG2);

stringstream ss; rectangle(frame, cv::Point(10, 2), cv::Point(100, 20), cv::Scalar(255, 255, 255), -1); ss << capture.get(CAP_PROP_POS_FRAMES); string frameNumberString = ss.str(); putText(frame, frameNumberString.c_str(), cv::Point(15, 15), FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0, 0));

        imshow("Frame", frame);
        imshow("FG Mask MOG 2", fgMaskMOG2);

pMOG2->apply(frame, fgMaskMOG2);

        imshow("Frame", frame);
        imshow("FG Mask MOG 2", fgMaskMOG2);

//get the frame number and write it on the current frame rectangle(frame, Point(10, 2), Point(100, 20), Scalar(255, 255, 255), -1); frameNumberCurrent << capture.get(CAP_PROP_POS_FRAMES); frameInfo << frameNumberCurrent.str() << "/" << frameNumberTotal.str(); putText(frame, frameInfo.str(), Point(15, 15), FONT_HERSHEY_SIMPLEX, 0.5, Scalar(0, 0, 0));

//get the input from the keyboard keyboard = waitKey(30);

/// variable declaration and initialization
// morphing related variables (initialized during each loop, since parameters are changable)
Mat kernelClose;
Mat kernelErode;
Mat kernelDilate;

// frame related variables
stringstream    frameNumberTotal;
stringstream    frameNumberCurrent;
stringstream    frameInfo;
int             frameWidth = 0,
                frameHeight = 0;

// contour related variables
vector<vector<Point>>   contours;
vector<Vec4i>           hierarchy;

// displaying related variables
int     r = 1;      // counter for row

//// video processing

frameNumberTotal << capture.get(CAP_PROP_FRAME_COUNT);
frameWidth = (int)capture.get(CV_CAP_PROP_FRAME_WIDTH);
frameHeight = (int)capture.get(CV_CAP_PROP_FRAME_HEIGHT);

if (frameEnd == 0)
    frameEnd = (int)capture.get(CAP_PROP_FRAME_COUNT);

// set video to defined start frame
capture.set(CAP_PROP_POS_FRAMES, frameStart);

// show control panel (enables modification of some variables in realtime)
showControlPanel();

while ((char)keyboard != 27) {
    // read the current frame
    if (!capture.read(frame)) {
        cerr << "Unable to read next frame." << endl;
        cerr << "Exiting..." << endl;
        exit(EXIT_FAILURE);
    }

    // convert original frame to grayscale if specified by constant
    if (SOURCE2GRAY)
        frameGray.copyTo(frame);

    // re-initialize objects with changable parameters
    if (kernelSizeClose > 0)
        kernelClose = getStructuringElement(KERNEL_TYPE, Size(kernelSizeClose, kernelSizeClose));
    if (kernelSizeErode > 0)
        kernelErode = getStructuringElement(KERNEL_TYPE, Size(kernelSizeErode, kernelSizeErode));
    if (kernelSizeDilate > 0)
        kernelDilate = getStructuringElement(KERNEL_TYPE, Size(kernelSizeDilate, kernelSizeDilate));

    }
        // apply threshold to MOG2 frame to get rid of shadows etc
        threshold(fgMaskMOG2, frameThreshold, thresholdMask, 255, THRESH_BINARY);

        // apply morphology operations to thresholded frame

        if (kernelSizeClose > 0)
            morphologyEx(frameMorphed, frameMorphed, MORPH_CLOSE, kernelClose);
        if (kernelSizeErode > 0)
            morphologyEx(frameMorphed, frameMorphed, MORPH_ERODE, kernelErode);
        if (kernelSizeDilate > 0)
            morphologyEx(frameMorphed, frameMorphed, MORPH_DILATE, kernelDilate);

        frameThreshold.copyTo(frameMorphed);

        // save previous masked frame as grayscale frame for later tracking
        if (!frameMasked.empty())
            cvtColor(frameMasked, frameGrayPrev, CV_RGB2GRAY);

        // apply morphed mask, but save before the morphed frame before
        frameMorphed.copyTo(frameContours);
        frameMorphed.copyTo(frameMasked);
        frame.copyTo(frameMasked, frameMasked);

        // save new masked frame as grayscale frame for later tracking, also save mask to KltTracking frame
        cvtColor(frameMasked, frameGray, CV_RGB2GRAY);
        frameMasked.copyTo(frameKltTracking);

        // draw and fill contours
        findContours(frameContours, contours, hierarchy, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
        setIdentity(frameContours, Scalar(0, 0, 0));
        drawContours(frameContours, contours, -1, Scalar(255, 255, 255), -1);

        // draws rectangle around identified contours 
        drawBoundingBoxes(contours, frameMasked);

        // prepare for and apply tracking algorithm (use grayscale copies of masked frame)
        if (!frameGrayPrev.empty() && TRACKING)
            applyKltTracking(frameGray, frameGrayPrev, frameKltTracking, contours);

        /// show the different frames (from last to first row, to overlay windows bar)
        resizeFrames();             // resize all frames

    if (SHOW_ALL_OPERATION_RESULTS) {
        // row 3
        r = 3;
        showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);

        // row 2
        r = 2;
        showFrame("Morphed", frameMorphed, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
        showFrame("Contours", frameContours, frameWidth, frameHeight, 2 + (r - 1)*ELEMENTS_PER_ROW);
        showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 3 + (r - 1)*ELEMENTS_PER_ROW);

        // row 1
        r = 1;
        showFrame("Original", frame, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
        showFrame("Foreground (MOG2)", fgMaskMOG2, frameWidth, frameHeight, 2 + (r - 1)*ELEMENTS_PER_ROW);
        showFrame("Threshold", frameThreshold, frameWidth, frameHeight, 3 + (r - 1)*ELEMENTS_PER_ROW);

    }
    else {
        // row 1
        r = 1;
        showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
        showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 2 + (r - 1)*ELEMENTS_PER_ROW);
    }

    // shows image for the specified amount of milliseconds 
    keyboard = waitKey(30);

    switch (keyboard) {
    case 'r':
        resetFeatures = true;       // reset on pressed 'r'
        break;
    }

    // loop through the video if end is reached and loop is set
    if (LOOP_VIDEO && (frameNumberCurrent.str().compare(std::to_string(frameEnd)) == 0))
        capture.set(CAP_PROP_POS_FRAMES, frameStart);

    // ensure that new feature points are initialized every INIT_INTERVAL frames for the tracker
    if (std::stoi(frameNumberCurrent.str()) % INIT_INTERVAL == 0 && trackedObjects.size() > 0)
        resetFeatures = true;

    // cleaning operation: clean string streams
    frameNumberCurrent.str("");
    frameInfo.str("");
}

// cleaning operation: release capture object
capture.release();

    //imshow("FG Mask MOG 2", fgMaskMOG2);

    //get the input from the keyboard
    //keyboard = waitKey(30);
//}
//delete capture object
////capture.release();

}

char* videoChooser() {

// variable declaration
char*   videoFile = "dog.avi";  // saves name of selected video file

// chooses video file depending on chosen video number
switch (VIDEO_NUMBER) {
case 1:
    videoFile = "visiontraffic.avi";
    frameStart = 90;
    frameEnd = 440;
    break;

case 2:
    videoFile = "viptraffic.avi";
    break;

case 3:
    videoFile = "highwayII_raw.avi";
    break;

case 4:
    videoFile = "dispersed.ogv";
    break;

case 5:
    videoFile = "highspeed.ogv";
    break;

case 6:
    videoFile = "hightraffic-dark.ogv";
    break;

case 7:
    videoFile = "hightraffic-occlude.ogv";
    break;

case 8:
    videoFile = "threecars.ogv";
    break;

case 9:
    videoFile = "turntraffic-dark.ogv";
    break;

case 10:
    videoFile = "visor_1261565143617_cesta1.wmv";
    break;

default:
    break;
}

return videoFile;

}

// create y-Offset if defince to make titles of windows readable
if (!WINDOW_NAMES_READABLE) {
    yOffset += -32;
}
else if (position > ELEMENTS_PER_ROW) {
    yOffset += 32;
}

// create window and display defined frame

cv::namedWindow(windowName);
cv::imshow(windowName, frame);

// resize stats for position calculation, if defined
if (scale != 10) {
    // filter for the case, that it's zero, since range is not adjustable in opencv gui
    if (scale == 0)
        scale = 1;

    // calculate scalefactor from value of scale variable
    float scaleFactor = (float)scale / 10;

    // calculate new width and height
    width = (int)(width * scaleFactor);
    height = (int)(height * scaleFactor);
}

//// get positioning

// get x coordinates with width
if (((position - 1) % ELEMENTS_PER_ROW) != 0)
    x = ((position - 1) % ELEMENTS_PER_ROW) * width;

// get y coordinates with height
if (position > ELEMENTS_PER_ROW)
    y = (position - (position - 1) % ELEMENTS_PER_ROW) / ELEMENTS_PER_ROW * height;

// place window at calculated position
moveWindow(windowName, x, y + yOffset);

}

void resizeFrames() { //resize all frames, if defined if (scale != 10 && scale > 0) { // filter for the case, that it's zero, since range is not adjustable in opencv gui if (scale == 0) scale = 1;

    // calculate scalefactor from value of scale variable
    float scaleFactor = (float)scale / 10;

    // resize all frames
    resize(frame, frame, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
    resize(fgMaskMOG2, fgMaskMOG2, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
    resize(frameThreshold, frameThreshold, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
    resize(frameMorphed, frameMorphed, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
    resize(frameMasked, frameMasked, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
    resize(frameContours, frameContours, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
    resize(frameKltTracking, frameKltTracking, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
}

}

void drawBoundingBoxes(vector<vector> contours, Mat frame) { // variable declaration int amountHandledContours = 0; Scalar drawingColor = DEFAULT_DRAWING_COLOR;

// create bounding rect of object and draw on defined frame
for (vector<vector<Point>>::iterator itc = contours.begin(); itc != contours.end(); itc++) {
    // variable declaration and initialization
    Rect    boundingBox = boundingRect(Mat(*itc));
    Mat     overlay;

    // only handle rectangles with a minimum area
    if (boundingBox.area() >= CONTOUR_MIN_AREA) {
        if (contourArea(*itc) >= CONTOUR_MIN_AREA) {

            // get corresponding object, if there's one, to define the drawing color
            size_t* indexCorrespondingObject = getCorrespondingObject(*itc);
            if (indexCorrespondingObject != NULL)
                drawingColor = trackedObjects[*indexCorrespondingObject].getColor();

            // draw contours regardless of area
            drawContours(frame, contours, -1, drawingColor, 1);

            // handle bounding boxes (but only if they're bigger than defined minimum area)
            if (boundingBox.area() >= BBOX_MIN_AREA) {
                // copy frame to
                frame.copyTo(overlay);

                // draw rectangle 
                rectangle(overlay, boundingBox, drawingColor, 3);

                // draw test label if constant is set
                if (SHOW_LABELS)
                    rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y),
                    Point(boundingBox.x + boundingBox.width + 1,
                    boundingBox.y - LABEL_HEIGHT), drawingColor, -1);

                // combine transparent overlay of bounding rect with original frame
                cv::addWeighted(overlay, ALPHA, frame, 1 - ALPHA, 0, frame);

                // afterwards, put text with object name into the label, if constant is set
                if (SHOW_LABELS)
                    putText(frame, TEST_NAME, Point(boundingBox.x + 10, boundingBox.y - 2),
                    TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(0, 0, 0),
                    TEXT_THICKNESS, TEXT_LINE_TYPE, false);

                // count handled contours
                amountHandledContours++;
            }
        }
    }
}

// re-initialize feature tracker if new objects appeared in this frame
if (amountHandledContours > amountHandledContoursPrev) {
    resetFeatures = true;
    cout << amountHandledContours << "/" << amountHandledContoursPrev << endl;
}

amountHandledContoursPrev = amountHandledContours;

}

void showControlPanel() { // constant declaration const int WIDTH = 400, HEIGHT = 240;

const char* nameWindow = "Control Panel";
const char* nameTrackbarWindowScale = "Scale";
const char* nameTrackbarThreshold = "Threshold";
const char* nameTrackbarKernelSizeClose = "SizeClose";
const char* nameTrackbarKernelSizeErode = "SizeErode";
const char* nameTrackbarKernelSizeDilate = "SizeDilate";

// create window
namedWindow(nameWindow, CV_WINDOW_AUTOSIZE | CV_GUI_NORMAL);
resizeWindow(nameWindow, WIDTH, HEIGHT);

// window relating parameters
createTrackbar(nameTrackbarWindowScale, nameWindow, &scale, 20, NULL);
createTrackbar(nameTrackbarThreshold, nameWindow, &thresholdMask, THRESHOLD_MASK_MAX, NULL);

createTrackbar(nameTrackbarKernelSizeClose, nameWindow, &kernelSizeClose, KERNEL_SIZE_MAX, NULL);
createTrackbar(nameTrackbarKernelSizeErode, nameWindow, &kernelSizeErode, KERNEL_SIZE_MAX, NULL);
createTrackbar(nameTrackbarKernelSizeDilate, nameWindow, &kernelSizeDilate, KERNEL_SIZE_MAX, NULL);

}

/**

@function for klt tracking */ void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector> contours) {

/// variable declaration

vector newPoints;

//points[0].clear();    // could solve performance issue, but creates strange behaviour: 
// trackedObjects also reset every reset
//points[1].clear();

// initialize with all available points from tracked objects
for (std::vector<TrackedObject>::size_type l = 0; l != trackedObjects.size(); l++)
    for (std::vector<TrackedObject>::size_type j = 0; j != trackedObjects[l].getPoints().size(); j++) {
        points[0].push_back(trackedObjects[l].getPoints()[j]);
    }

} if (!points[0].empty() || trackedObjects.size() > 0) { // variable declaration vector status; vector err; size_t i, k;

// check if previous frame is existing, otherwise copy the current one
if (frameGrayPrev.empty())
    frameGray.copyTo(frameGrayPrev);

// calculate the optical flow with KLT
calcOpticalFlowPyrLK(frameGrayPrev, frameGray, points[0], points[1], status, err, winSize,
    3, termcrit, 0, 0.001);

/// iterate through all found points
for (i = k = 0; i < points[1].size(); i++) {

    // don't handle point, if there's no corresponding point from the last frame
    if (!status[i])
        continue;

    // otherwise save and draw point (shrink's point vector, resizes after the for loop)
    points[1][k++] = points[1][i];

    /// get corresponding contour and object, otherwise create new tracked object
    if (STATEFUL_TRACKING)
        trackObjects(contours, i);
    else
        circle(frameKltTracking, points[1][i], 6, DEFAULT_DRAWING_COLOR, 2, 8);
}

points[1].resize(k);

}

// handle the processed points handleProcessedPoints();

// save that initialization was done and not needed anymore, also add new points if (resetFeatures) { resetFeatures = false;

for (std::vector<Point2f>::size_type l = 0; l != newPoints.size(); l++)
    points[1].push_back(newPoints[l]);

}

// save the current values as the previous ones if (!resetFeatures) std::swap(points[1], points[0]);

cv::swap(frameGrayPrev, frameGrayPrev); }

/// get contour of point
for (m = 0; m != contours.size(); m++) {
    // checks if point is inside or on the edge of the contour
    containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false);

    // exit loop if corresponding contour is found
    if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) {
        foundContour = true;
        break;
    }
}

if (trackedObjects.size() > 0) {
    // check for each point in the tracked object if it already exists
    for (std::vector<TrackedObject>::size_type l = 0; l != trackedObjects.size(); l++) {
        size_t* containedInObject = NULL;

        if (trackedObjects[l].containsOldPoint(points[0][currentPoint]) != NULL) {
            // add the point (it will increase weight automatically instead)
            trackedObjects[l].addPoint(points[1][currentPoint], points[0][currentPoint]);
            foundObject = true;

            // update contour of object, if contour was not modified this frame yet
            bool foundContour = false;
            if (!trackedObjects[l].getContourStatus()) {
                std::vector<Point>::size_type m;
                for (m = 0; m != contours.size(); m++) {
                    // checks if point is inside or on the edge of the contour
                    containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false);

                    // exit loop if corresponding contour is found
                    if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) {
                        foundContour = true;
                        break;
                    }
                }

                // update corresponding contour, if one was found
                if (foundContour)
                    trackedObjects[l].setContour(contours[m]);
            }
        }

    }
}

// create new object with this point
if (!foundObject && foundContour) {
    size_t* containedInObject;
    containedInObject = getCorrespondingObject(contours[m]);

    if (containedInObject != NULL) {
        // contour belongs to an existing object, so add the point
        size_t containedInObjectVal = *containedInObject;

        // add points 
        if (trackedObjects[containedInObjectVal].getPoints().size() < MAX_COUNT_PER_CAR)
            trackedObjects[containedInObjectVal].addPoint(points[1][currentPoint], points[0][currentPoint]);

        foundObjectOfContour = true;
    }

    // create new tracked object, since there's no existing one with this contour
    if (!foundObjectOfContour) {
        char name[11] = "Object_";
        char objectNumber[4];
        sprintf(objectNumber, "%d", (int)(objectCounter + 1));
        strcat(name, objectNumber);

        TrackedObject* trObj = new TrackedObject(name);
        trObj->addPoint(points[1][currentPoint], points[0][currentPoint]);

        trObj->setContour(contours[m]);

        trackedObjects.push_back(*trObj);
        objectCounter++;
    }
}

}

size_t getCorrespondingObject(vector contour) { // variable declaration size_t position = NULL;

// find out if contour is assigned to existing object
if (trackedObjects.size() > 0 && contour.size() > 0) {
    for (std::vector<TrackedObject>::size_type i = 0; i != trackedObjects.size(); i++) {
        if (trackedObjects[i].getContour().size() == contour.size()) {
            double comparisonValue;
            comparisonValue = matchShapes(trackedObjects[i].getContour(), contour, CV_CONTOURS_MATCH_I1, NULL);
            if (comparisonValue == 0) {
                position = &i;
                break;
            }
        }
    }
}

return position;

}

void handleProcessedPoints() { // get original frame, if constant defines, that results should be shown in original if (TRACKING_RESULTS_IN_ORIGINAL) frame.copyTo(frameKltTracking);

if (trackedObjects.size() > 0) {
    for (std::vector<TrackedObject>::size_type i = 0; i != trackedObjects.size(); i++) {

        // process all points according to the status
        trackedObjects[i].processStatus();

        if (trackedObjects[i].getPoints().size() <= MIN_POINTS_TRACKED_OBJECT) {
            trackedObjects.erase(trackedObjects.begin() + i);
            i--;
            continue;
        }

        /// draw bounding box 
        Rect    boundingBox = boundingRect(trackedObjects[i].getContour());
        Mat overlay;
        frameKltTracking.copyTo(overlay);

        /// draw rectangle and label
        rectangle(overlay, boundingBox, trackedObjects[i].getColor(), 3);
        rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y),
            Point(boundingBox.x + boundingBox.width + 1,
            boundingBox.y - LABEL_HEIGHT), trackedObjects[i].getColor(), -1);

        // combine transparent overlay of bounding rect with original frame
        cv::addWeighted(overlay, ALPHA, frameKltTracking, 1 - ALPHA, 0, frameKltTracking);

        // afterwards, put text with object name into the label, if constant is set
        putText(frameKltTracking, trackedObjects[i].getName(), Point(boundingBox.x + 10, boundingBox.y - 2),
            TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(200, 200, 200),
            TEXT_THICKNESS, TEXT_LINE_TYPE, false);

        /// draw all feature points in the color of the corresponding object
        for (std::vector<Point2f>::size_type j = 0; j != trackedObjects[i].getPoints().size(); j++) {
            // handle weight of point
            int w = trackedObjects[i].getWeight()[j];
            int size = (int)trackedObjects[i].getPoints().size();

            if (w != -1) {  // point not found
                // draw filled circle for new points
                if (w == 1)
                    w = -1;
                else if (w >= HEATMAP_RANGE)
                    w = HEATMAP_RANGE;

                if (w != -1)
                    circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, getHeatMapColor(1 - (float)w / (float)HEATMAP_RANGE), -1, 8);
                circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, trackedObjects[i].getColor(), 2, 8);
            }
        }
    }
}

}

int idx1;        // |-- Our desired color will be between these two indexes in "color".
int idx2;        // |
float fractBetween = 0;  // Fraction between "idx1" and "idx2" where our value is.

if (value <= 0) { idx1 = idx2 = 0; }    // accounts for an input <=0
else if (value >= 1) { idx1 = idx2 = NUM_COLORS - 1; }    // accounts for an input >=0
else
{
    value = value * (NUM_COLORS - 1);       
    idx1 = (int)floor(value);               // Our desired color will be after this index.
    idx2 = idx1 + 1;                        // ... and before this index (inclusive).
    fractBetween = value - float(idx1);     // Distance between the two indexes (0-1).
}

red = ((color[idx2][0] - color[idx1][0])*fractBetween + color[idx1][0]) * 255;
green = ((color[idx2][1] - color[idx1][1])*fractBetween + color[idx1][1]) * 255;
blue = ((color[idx2][2] - color[idx1][2])*fractBetween + color[idx1][2]) * 255;

return Scalar(red, green, blue);

}

Hi, still not clear what do you want, this code is for MOG2, it uses MOG2, and if it works, than no bugs here.

2017-08-23 7:10 GMT+03:00 dulithaaaa notifications@github.com:

Thank you very much sir,sir i have code can you find what is bug in the code please,it works only to MOG2

On Wednesday, August 23, 2017 9:16 AM, Sergey Nuzhny < notifications@github.com> wrote:

Sir,I have a code of tracking i attached that file.it worked only to MOG2,could you please look at that code and tell me what modification needs to be required. Hi! I don't know what do you want. How can I advise you? This project - Multitarget-tracker contains full stack of algorithms for motion detection. I can tell you how it works.

You need CMake gui application, start it.

Select folder Mulitarget-tracker (with CMakeLists.txt file).

Select subfolder folder build.

Push a button Configure, select your MSVC compler. Wait.

If OpenCV wasn't founded automatic then select OpenCV_DIR manually. Push a button Configure again.

Push a button Generate.

Now the folder build contains a solution file (.sln). Lets open it in Visual studio! The end. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

include "opencv2/imgcodecs.hpp"

include "opencv2/imgproc.hpp"

include "opencv2/videoio.hpp"

include <opencv2/highgui.hpp>

include <opencv2/video.hpp>

include

include

include

include

include "TrackedObject.h"

using namespace cv; using namespace std;

// Global constant

const bool SHOW_ALL_OPERATION_RESULTS = true; // defines if different operation results should be shown const bool WINDOW_NAMES_READABLE = false; // if set, creates y-offset to make window names readable const int ELEMENTS_PER_ROW = 3; // defines how much windows are shown within a row const bool TRACKING_RESULTS_IN_ORIGINAL = false; // shows tracking results in original frame

// video related constants const int VIDEO_NUMBER = 1; // 1:10, choose different number for different video const bool LOOP_VIDEO = true; // defines if video should be looped infinitely

const char* VIDEO_DIRECTORY_PATH = ".\detect.avi\"; // define path to video directory

const bool SOURCE2GRAY = false; // defines, if source frame should be converted to gray

// foreground segmentation postprocessing related constants const int HISTORY = 1000; // MOG2 parameter: amount of considered frames const double VAR_THRESHOLD = 20; // MOG2 parameter: threshold for degree of minimum variance const bool DETECT_SHADOWS = false; // MOG2 parameter: defines if shadows should be detected

int thresholdMask = 100; // defines start value of threshold for // binarization of foreground frame const int THRESHOLD_MASK_MAX = 255; // defines maximum value for threshold

// morphing related constants const int KERNEL_TYPE = MORPH_ELLIPSE; // choose: MORPH_RECT, MORPH_CROSS, MORPH_ELLIPSE int kernelSizeClose = 5; // defines start value for kernel size of close op. int kernelSizeErode = 5; // defines start value for kernel size of erode op. int kernelSizeDilate = 5; // defines start value for kernel size of dilate op. const int KERNEL_SIZE_MAX = 20; // defines maximum value for threshold

// drawing related constants const Scalar DEFAULT_DRAWING_COLOR = Scalar(0, 255, 0); // defines default color of bounding boxes and labels const int CONTOUR_MIN_AREA = 500; // defines minimum area of contours const int BBOX_MIN_AREA = 3000; // defines minimum area of bounding boxes const bool SHOW_LABELS = false; // shows labels of object adjusted to the bounding box const int HEATMAP_RANGE = 25; // defines size of different colors in heat map // and threshold (color of highest weight)

// bounding box and label related constants const float ALPHA = 0.5; // defines opacity of rectangle and label const int LABEL_HEIGHT = 20; // defines height of label const char* TEST_NAME = "Car"; // defines name for test labels

const int TEXT_FONT_FACE = 2; // defines font of label text const int TEXT_THICKNESS = 2; // defines thickness of label text const int TEXT_LINE_TYPE = 0; // defines line type of label text const double TEXT_FONT_SCALE = 0.7; // defines scale of label text

// tracking related constants const bool TRACKING = true; // defines if tracking is active const bool STATEFUL_TRACKING = true; // defines if stateful tracking with TrackedObjects is active const int MAX_COUNT_TOTAL = 150; // defines max no. of tracked points in total const int MAX_COUNT_PER_CAR = 25; // defines max no. of tracked points per car const int MIN_POINTS_TRACKED_OBJECT = 3; // defines min no. of points for tracked objects const int INIT_INTERVAL = 15; // defines, after how many frames the tracker should initialize new points const TermCriteria termcrit(TermCriteria::COUNT | TermCriteria::EPS, 20, 0.03); // defines criteria for klt tracker const Size subPixWinSize(10, 10); const Size winSize(31, 31);

/// Global variables

// window related variables int scale = 10; // scales windows (and mats) after processing before displaying // (10 is equal to standard size, 20 to doubled size)

// video related variables int frameStart = 1; // defines frame to start (set in video chooser) int frameEnd = 0; // defines frame to end (set in video chooser)

// frame related variables

Mat frame; //current frame Mat fgMaskMOG2; //fg mask fg mask generated by MOG2 method Mat frameGray; // current frame in grayscale Mat frameGrayPrev; // previous frame in grayscale Mat frameThreshold; // frame with applied threshold (to elimante shadows etc.) Mat frameMorphed; // morphed frame (to elimante noise) Mat frameMasked; // morphed frame applied as a mask to the original frame Mat frameContours; // drawn contours Mat frameKltTracking; // klt tracking lanes

Ptr pMOG2; //MOG2 Background subtractor

// tracking related variables bool resetFeatures = false; Point2f point; vector points[2]; int amountHandledContoursPrev = 0;

vector trackedObjects; int objectCounter;

int keyboard; //input from keyboard

/// Function Headers

char* videoChooser();

void processVideo(char* videoFilename);

void showFrame(char* windowName, Mat frame, int width,int height, int position);

void resizeFrames();

void drawBoundingBoxes(vector<vector> contours, Mat frame);

void showControlPanel();

void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector> contours);

void trackObjects(vector<vector> contours, size_t currentPoint);

size_t* getCorrespondingObject(vector contour);

void handleProcessedPoints();

Scalar getHeatMapColor(float value);

void processVideo(char videoFilename); void processImages(char firstFrameFilename);

char* VIDEO_FILE = "visiontraffic.avi";

int main(int argc, char* argv[]) {

char* videoPath = new char[100];

/// initialize variables // define path of video file strcpy(videoPath, VIDEO_DIRECTORY_PATH); strcat(videoPath, videoChooser());

//create GUI windows namedWindow("Original"); namedWindow("Foreground (MOG2)"); namedWindow("Threshold"); namedWindow("Morphed"); namedWindow("Contours"); namedWindow("Masked Original Frame"); namedWindow("KLT Tracking");

//create Background Subtractor objects pMOG2 = createBackgroundSubtractorMOG2(HISTORY, VAR_THRESHOLD, DETECT_SHADOWS); //MOG2 approach

//input data coming from a video processVideo(VIDEO_FILE); //destroy GUI windows destroyAllWindows(); return EXIT_SUCCESS; }

void processVideo(char* videoFilename) {

//create the capture object VideoCapture capture(videoFilename); if (!capture.isOpened()) { //error in opening the video input cerr << "Unable to open video file: " << videoFilename << endl; exit(EXIT_FAILURE); }

///read input data. ESC or 'q' for quitting while ((char)keyboard != 'q' && (char)keyboard != 27) { //read the current frame if (!capture.read(frame)) { cerr << "Unable to read next frame." << endl; cerr << "Exiting..." << endl; exit(EXIT_FAILURE); }

//update the background model pMOG2->apply(frame, fgMaskMOG2);

stringstream ss; rectangle(frame, cv::Point(10, 2), cv::Point(100, 20), cv::Scalar(255, 255, 255), -1); ss << capture.get(CAP_PROP_POS_FRAMES); string frameNumberString = ss.str(); putText(frame, frameNumberString.c_str(), cv::Point(15, 15), FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0, 0));

imshow("Frame", frame); imshow("FG Mask MOG 2", fgMaskMOG2);

pMOG2->apply(frame, fgMaskMOG2);

imshow("Frame", frame); imshow("FG Mask MOG 2", fgMaskMOG2);

//get the frame number and write it on the current frame rectangle(frame, Point(10, 2), Point(100, 20), Scalar(255, 255, 255), -1); frameNumberCurrent << capture.get(CAP_PROP_POS_FRAMES); frameInfo << frameNumberCurrent.str() << "/" << frameNumberTotal.str(); putText(frame, frameInfo.str(), Point(15, 15), FONT_HERSHEY_SIMPLEX, 0.5, Scalar(0, 0, 0));

//get the input from the keyboard keyboard = waitKey(30);

/// variable declaration and initialization // morphing related variables (initialized during each loop, since parameters are changable) Mat kernelClose; Mat kernelErode; Mat kernelDilate;

// frame related variables stringstream frameNumberTotal; stringstream frameNumberCurrent; stringstream frameInfo; int frameWidth = 0, frameHeight = 0;

// contour related variables vector<vector> contours; vector hierarchy;

// displaying related variables int r = 1; // counter for row

//// video processing

frameNumberTotal << capture.get(CAP_PROP_FRAME_COUNT); frameWidth = (int)capture.get(CV_CAP_PROP_FRAME_WIDTH); frameHeight = (int)capture.get(CV_CAP_PROP_FRAME_HEIGHT);

if (frameEnd == 0) frameEnd = (int)capture.get(CAP_PROP_FRAME_COUNT);

// set video to defined start frame capture.set(CAP_PROP_POS_FRAMES, frameStart);

// show control panel (enables modification of some variables in realtime) showControlPanel();

while ((char)keyboard != 27) { // read the current frame if (!capture.read(frame)) { cerr << "Unable to read next frame." << endl; cerr << "Exiting..." << endl; exit(EXIT_FAILURE); }

// convert original frame to grayscale if specified by constant if (SOURCE2GRAY) frameGray.copyTo(frame);

// re-initialize objects with changable parameters if (kernelSizeClose > 0) kernelClose = getStructuringElement(KERNEL_TYPE, Size(kernelSizeClose, kernelSizeClose)); if (kernelSizeErode > 0) kernelErode = getStructuringElement(KERNEL_TYPE, Size(kernelSizeErode, kernelSizeErode)); if (kernelSizeDilate > 0) kernelDilate = getStructuringElement(KERNEL_TYPE, Size(kernelSizeDilate, kernelSizeDilate));

} // apply threshold to MOG2 frame to get rid of shadows etc threshold(fgMaskMOG2, frameThreshold, thresholdMask, 255, THRESH_BINARY);

// apply morphology operations to thresholded frame

if (kernelSizeClose > 0) morphologyEx(frameMorphed, frameMorphed, MORPH_CLOSE, kernelClose); if (kernelSizeErode > 0) morphologyEx(frameMorphed, frameMorphed, MORPH_ERODE, kernelErode); if (kernelSizeDilate > 0) morphologyEx(frameMorphed, frameMorphed, MORPH_DILATE, kernelDilate);

frameThreshold.copyTo(frameMorphed);

// save previous masked frame as grayscale frame for later tracking if (!frameMasked.empty()) cvtColor(frameMasked, frameGrayPrev, CV_RGB2GRAY);

// apply morphed mask, but save before the morphed frame before frameMorphed.copyTo(frameContours); frameMorphed.copyTo(frameMasked); frame.copyTo(frameMasked, frameMasked);

// save new masked frame as grayscale frame for later tracking, also save mask to KltTracking frame cvtColor(frameMasked, frameGray, CV_RGB2GRAY); frameMasked.copyTo(frameKltTracking);

// draw and fill contours findContours(frameContours, contours, hierarchy, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE); setIdentity(frameContours, Scalar(0, 0, 0)); drawContours(frameContours, contours, -1, Scalar(255, 255, 255), -1);

// draws rectangle around identified contours drawBoundingBoxes(contours, frameMasked);

// prepare for and apply tracking algorithm (use grayscale copies of masked frame) if (!frameGrayPrev.empty() && TRACKING) applyKltTracking(frameGray, frameGrayPrev, frameKltTracking, contours);

/// show the different frames (from last to first row, to overlay windows bar) resizeFrames(); // resize all frames

if (SHOW_ALL_OPERATION_RESULTS) { // row 3 r = 3; showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);

// row 2 r = 2; showFrame("Morphed", frameMorphed, frameWidth, frameHeight, 1 + (r - 1)ELEMENTS_PER_ROW); showFrame("Contours", frameContours, frameWidth, frameHeight, 2 + (r - 1)ELEMENTS_PER_ROW); showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 3

(r - 1)*ELEMENTS_PER_ROW);

// row 1 r = 1; showFrame("Original", frame, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW); showFrame("Foreground (MOG2)", fgMaskMOG2, frameWidth, frameHeight, 2 + (r

1)ELEMENTS_PER_ROW); showFrame("Threshold", frameThreshold, frameWidth, frameHeight, 3 + (r - 1)ELEMENTS_PER_ROW);

} else { // row 1 r = 1; showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 1

(r - 1)ELEMENTS_PER_ROW); showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 2 + (r - 1)ELEMENTS_PER_ROW); }

// shows image for the specified amount of milliseconds keyboard = waitKey(30);

switch (keyboard) { case 'r': resetFeatures = true; // reset on pressed 'r' break; }

// loop through the video if end is reached and loop is set if (LOOP_VIDEO && (frameNumberCurrent.str().compare(std::to_string(frameEnd)) == 0)) capture.set(CAP_PROP_POS_FRAMES, frameStart);

// ensure that new feature points are initialized every INIT_INTERVAL frames for the tracker if (std::stoi(frameNumberCurrent.str()) % INIT_INTERVAL == 0 && trackedObjects.size() > 0) resetFeatures = true;

// cleaning operation: clean string streams frameNumberCurrent.str(""); frameInfo.str(""); }

// cleaning operation: release capture object capture.release();

//imshow("FG Mask MOG 2", fgMaskMOG2);

//get the input from the keyboard //keyboard = waitKey(30); //} //delete capture object ////capture.release(); }

char* videoChooser() {

// variable declaration char* videoFile = "dog.avi"; // saves name of selected video file

// chooses video file depending on chosen video number switch (VIDEO_NUMBER) { case 1: videoFile = "visiontraffic.avi"; frameStart = 90; frameEnd = 440; break;

case 2: videoFile = "viptraffic.avi"; break;

case 3: videoFile = "highwayII_raw.avi"; break;

case 4: videoFile = "dispersed.ogv"; break;

case 5: videoFile = "highspeed.ogv"; break;

case 6: videoFile = "hightraffic-dark.ogv"; break;

case 7: videoFile = "hightraffic-occlude.ogv"; break;

case 8: videoFile = "threecars.ogv"; break;

case 9: videoFile = "turntraffic-dark.ogv"; break;

case 10: videoFile = "visor_1261565143617_cesta1.wmv"; break;

default: break; }

return videoFile; }

void showFrame(char* windowName, Mat frame, int width, int height, int position) { // variable declaration and initialization int x = 0, // x-coordinates for position y = 0, // y-coordinates for position yOffset = 30; // offset for y-coordinates, since windows bar does not count into frame height // strategy for y-offset: create windows from last row to first to overlay the bar

// create y-Offset if defince to make titles of windows readable if (!WINDOW_NAMES_READABLE) { yOffset += -32; } else if (position > ELEMENTS_PER_ROW) { yOffset += 32; }

// create window and display defined frame

cv::namedWindow(windowName); cv::imshow(windowName, frame);

// resize stats for position calculation, if defined if (scale != 10) { // filter for the case, that it's zero, since range is not adjustable in opencv gui if (scale == 0) scale = 1;

// calculate scalefactor from value of scale variable float scaleFactor = (float)scale / 10;

// calculate new width and height width = (int)(width scaleFactor); height = (int)(height scaleFactor); }

//// get positioning

// get x coordinates with width if (((position - 1) % ELEMENTS_PER_ROW) != 0) x = ((position - 1) % ELEMENTS_PER_ROW) * width;

// get y coordinates with height if (position > ELEMENTS_PER_ROW) y = (position - (position - 1) % ELEMENTS_PER_ROW) / ELEMENTS_PER_ROW * height;

// place window at calculated position moveWindow(windowName, x, y + yOffset); }

void resizeFrames() { //resize all frames, if defined if (scale != 10 && scale > 0) { // filter for the case, that it's zero, since range is not adjustable in opencv gui if (scale == 0) scale = 1;

// calculate scalefactor from value of scale variable float scaleFactor = (float)scale / 10;

// resize all frames resize(frame, frame, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(fgMaskMOG2, fgMaskMOG2, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameThreshold, frameThreshold, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameMorphed, frameMorphed, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameMasked, frameMasked, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameContours, frameContours, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameKltTracking, frameKltTracking, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); } }

void drawBoundingBoxes(vector<vector> contours, Mat frame) { // variable declaration int amountHandledContours = 0; Scalar drawingColor = DEFAULT_DRAWING_COLOR;

// create bounding rect of object and draw on defined frame for (vector<vector>::iterator itc = contours.begin(); itc != contours.end(); itc++) { // variable declaration and initialization Rect boundingBox = boundingRect(Mat(*itc)); Mat overlay;

// only handle rectangles with a minimum area if (boundingBox.area() >= CONTOUR_MIN_AREA) { if (contourArea(*itc) >= CONTOUR_MIN_AREA) {

// get corresponding object, if there's one, to define the drawing color size_t indexCorrespondingObject = getCorrespondingObject(itc); if (indexCorrespondingObject != NULL) drawingColor = trackedObjects[*indexCorrespondingObject].getColor();

// draw contours regardless of area drawContours(frame, contours, -1, drawingColor, 1);

// handle bounding boxes (but only if they're bigger than defined minimum area) if (boundingBox.area() >= BBOX_MIN_AREA) { // copy frame to frame.copyTo(overlay);

// draw rectangle rectangle(overlay, boundingBox, drawingColor, 3);

// draw test label if constant is set if (SHOW_LABELS) rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y), Point(boundingBox.x + boundingBox.width + 1, boundingBox.y - LABEL_HEIGHT), drawingColor, -1);

// combine transparent overlay of bounding rect with original frame cv::addWeighted(overlay, ALPHA, frame, 1 - ALPHA, 0, frame);

// afterwards, put text with object name into the label, if constant is set if (SHOW_LABELS) putText(frame, TEST_NAME, Point(boundingBox.x + 10, boundingBox.y - 2), TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(0, 0, 0), TEXT_THICKNESS, TEXT_LINE_TYPE, false);

// count handled contours amountHandledContours++; } } } }

// re-initialize feature tracker if new objects appeared in this frame if (amountHandledContours > amountHandledContoursPrev) { resetFeatures = true; cout << amountHandledContours << "/" << amountHandledContoursPrev << endl; }

amountHandledContoursPrev = amountHandledContours; }

void showControlPanel() { // constant declaration const int WIDTH = 400, HEIGHT = 240;

const char nameWindow = "Control Panel"; const char nameTrackbarWindowScale = "Scale"; const char nameTrackbarThreshold = "Threshold"; const char nameTrackbarKernelSizeClose = "SizeClose"; const char nameTrackbarKernelSizeErode = "SizeErode"; const char nameTrackbarKernelSizeDilate = "SizeDilate";

// create window namedWindow(nameWindow, CV_WINDOW_AUTOSIZE | CV_GUI_NORMAL); resizeWindow(nameWindow, WIDTH, HEIGHT);

// window relating parameters createTrackbar(nameTrackbarWindowScale, nameWindow, &scale, 20, NULL); createTrackbar(nameTrackbarThreshold, nameWindow, &thresholdMask, THRESHOLD_MASK_MAX, NULL);

createTrackbar(nameTrackbarKernelSizeClose, nameWindow, &kernelSizeClose, KERNEL_SIZE_MAX, NULL); createTrackbar(nameTrackbarKernelSizeErode, nameWindow, &kernelSizeErode, KERNEL_SIZE_MAX, NULL); createTrackbar(nameTrackbarKernelSizeDilate, nameWindow, &kernelSizeDilate, KERNEL_SIZE_MAX, NULL); }

/**

@function for klt tracking */ void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector> contours) {

/// variable declaration

vector newPoints;

/// calculate features and then the optical flow if (resetFeatures) { // automatic initialization goodFeaturesToTrack(frameGray, newPoints, MAX_COUNT_TOTAL, 0.01, 10, Mat(), 3, 0, 0.04); cornerSubPix(frameGray, newPoints, subPixWinSize, Size(-1, -1), termcrit);

//points[0].clear(); // could solve performance issue, but creates strange behaviour: // trackedObjects also reset every reset //points[1].clear();

// initialize with all available points from tracked objects for (std::vector::size_type l = 0; l != trackedObjects.size(); l++) for (std::vector::size_type j = 0; j != trackedObjects[l].getPoints().size(); j++) { points[0].push_back(trackedObjects[l].getPoints()[j]); } } if (!points[0].empty() || trackedObjects.size() > 0) { // variable declaration vector status; vector err; size_t i, k;

// check if previous frame is existing, otherwise copy the current one if (frameGrayPrev.empty()) frameGray.copyTo(frameGrayPrev);

// calculate the optical flow with KLT calcOpticalFlowPyrLK(frameGrayPrev, frameGray, points[0], points[1], status, err, winSize, 3, termcrit, 0, 0.001);

/// iterate through all found points for (i = k = 0; i < points[1].size(); i++) {

// don't handle point, if there's no corresponding point from the last frame if (!status[i]) continue;

// otherwise save and draw point (shrink's point vector, resizes after the for loop) points[1][k++] = points[1][i];

/// get corresponding contour and object, otherwise create new tracked object if (STATEFUL_TRACKING) trackObjects(contours, i); else circle(frameKltTracking, points[1][i], 6, DEFAULT_DRAWING_COLOR, 2, 8); }

points[1].resize(k); }

// handle the processed points handleProcessedPoints();

// save that initialization was done and not needed anymore, also add new points if (resetFeatures) { resetFeatures = false;

for (std::vector::size_type l = 0; l != newPoints.size(); l++) points[1].push_back(newPoints[l]); }

// save the current values as the previous ones if (!resetFeatures) std::swap(points[1], points[0]);

cv::swap(frameGrayPrev, frameGrayPrev); }

void trackObjects(vector<vector> contours, size_t currentPoint) { /// variable declaration int containedInContour; bool foundObject = false; bool foundContour = false; bool foundObjectOfContour = false; vector::size_type m; // iterator for contours size_t* correspondingObject = NULL; // pointer for corresponding object

/// get contour of point for (m = 0; m != contours.size(); m++) { // checks if point is inside or on the edge of the contour containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false);

// exit loop if corresponding contour is found if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) { foundContour = true; break; } }

if (trackedObjects.size() > 0) { // check for each point in the tracked object if it already exists for (std::vector::size_type l = 0; l != trackedObjects.size(); l++) { size_t* containedInObject = NULL;

if (trackedObjects[l].containsOldPoint(points[0][currentPoint]) != NULL) { // add the point (it will increase weight automatically instead) trackedObjects[l].addPoint(points[1][currentPoint], points[0][currentPoint]); foundObject = true;

// update contour of object, if contour was not modified this frame yet bool foundContour = false; if (!trackedObjects[l].getContourStatus()) { std::vector::size_type m; for (m = 0; m != contours.size(); m++) { // checks if point is inside or on the edge of the contour containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false);

// exit loop if corresponding contour is found if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) { foundContour = true; break; } }

// update corresponding contour, if one was found if (foundContour) trackedObjects[l].setContour(contours[m]); } }

} }

// create new object with this point if (!foundObject && foundContour) { size_t* containedInObject; containedInObject = getCorrespondingObject(contours[m]);

if (containedInObject != NULL) { // contour belongs to an existing object, so add the point size_t containedInObjectVal = *containedInObject;

// add points if (trackedObjects[containedInObjectVal].getPoints().size() < MAX_COUNT_PER_CAR) trackedObjects[containedInObjectVal].addPoint(points[1][currentPoint], points[0][currentPoint]);

foundObjectOfContour = true; }

// create new tracked object, since there's no existing one with this contour if (!foundObjectOfContour) { char name[11] = "Object_"; char objectNumber[4]; sprintf(objectNumber, "%d", (int)(objectCounter + 1)); strcat(name, objectNumber);

TrackedObject* trObj = new TrackedObject(name); trObj->addPoint(points[1][currentPoint], points[0][currentPoint]);

trObj->setContour(contours[m]);

trackedObjects.push_back(*trObj); objectCounter++; } } }

size_t getCorrespondingObject(vector contour) { // variable declaration size_t position = NULL;

// find out if contour is assigned to existing object if (trackedObjects.size() > 0 && contour.size() > 0) { for (std::vector::size_type i = 0; i != trackedObjects.size(); i++) { if (trackedObjects[i].getContour().size() == contour.size()) { double comparisonValue; comparisonValue = matchShapes(trackedObjects[i].getContour(), contour, CV_CONTOURS_MATCH_I1, NULL); if (comparisonValue == 0) { position = &i; break; } } } }

return position; }

void handleProcessedPoints() { // get original frame, if constant defines, that results should be shown in original if (TRACKING_RESULTS_IN_ORIGINAL) frame.copyTo(frameKltTracking);

if (trackedObjects.size() > 0) { for (std::vector::size_type i = 0; i != trackedObjects.size(); i++) {

// process all points according to the status trackedObjects[i].processStatus();

if (trackedObjects[i].getPoints().size() <= MIN_POINTS_TRACKED_OBJECT) { trackedObjects.erase(trackedObjects.begin() + i); i--; continue; }

/// draw bounding box Rect boundingBox = boundingRect(trackedObjects[i].getContour()); Mat overlay; frameKltTracking.copyTo(overlay);

/// draw rectangle and label rectangle(overlay, boundingBox, trackedObjects[i].getColor(), 3); rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y), Point(boundingBox.x + boundingBox.width + 1, boundingBox.y - LABEL_HEIGHT), trackedObjects[i].getColor(), -1);

// combine transparent overlay of bounding rect with original frame cv::addWeighted(overlay, ALPHA, frameKltTracking, 1 - ALPHA, 0, frameKltTracking);

// afterwards, put text with object name into the label, if constant is set putText(frameKltTracking, trackedObjects[i].getName(), Point(boundingBox.x

10, boundingBox.y - 2), TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(200, 200, 200), TEXT_THICKNESS, TEXT_LINE_TYPE, false);

/// draw all feature points in the color of the corresponding object for (std::vector::size_type j = 0; j != trackedObjects[i].getPoints().size(); j++) { // handle weight of point int w = trackedObjects[i].getWeight()[j]; int size = (int)trackedObjects[i].getPoints().size();

if (w != -1) { // point not found // draw filled circle for new points if (w == 1) w = -1; else if (w >= HEATMAP_RANGE) w = HEATMAP_RANGE;

if (w != -1) circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, getHeatMapColor(1 - (float)w / (float)HEATMAP_RANGE), -1, 8); circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, trackedObjects[i].getColor(), 2, 8); } } } } }

Scalar getHeatMapColor(float value) { float red, green, blue; const int NUM_COLORS = 4; static float color[NUM_COLORS][3] = { { 0, 0, 1 }, { 0, 1, 0 }, { 1, 1, 0 }, { 1, 0, 0 } }; // A static array of 4 colors: (blue, green, yellow, red) using {r,g,b} for each.

int idx1; // |-- Our desired color will be between these two indexes in "color". int idx2; // | float fractBetween = 0; // Fraction between "idx1" and "idx2" where our value is.

if (value <= 0) { idx1 = idx2 = 0; } // accounts for an input <=0 else if (value >= 1) { idx1 = idx2 = NUM_COLORS - 1; } // accounts for an input >=0 else { value = value * (NUM_COLORS - 1); idx1 = (int)floor(value); // Our desired color will be after this index. idx2 = idx1 + 1; // ... and before this index (inclusive). fractBetween = value - float(idx1); // Distance between the two indexes (0-1). }

red = ((color[idx2][0] - color[idx1][0])fractBetween + color[idx1][0]) 255; green = ((color[idx2][1] - color[idx1][1])*fractBetween + color[idx1][1])

255; blue = ((color[idx2][2] - color[idx1][2])fractBetween + color[idx1][2]) 255;

return Scalar(red, green, blue);

}

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Smorodov/Multitarget-tracker/issues/43#issuecomment-324215822, or mute the thread https://github.com/notifications/unsubscribe-auth/ABu3jdDAJ3O_HP4btBfv1NveFOQHM5plks5sa6YhgaJpZM4O-EOi .

Sir,I need to track the moving objects specially animals,Do you have static background animal moving videos which are more supportive for tracking ,could you send me please sir

On Wednesday, August 23, 2017 3:12 PM, Andrey Smorodov <notifications@github.com> wrote:

Hi, still not clear what do you want, this code is for MOG2, it uses MOG2, and if it works, than no bugs here.

2017-08-23 7:10 GMT+03:00 dulithaaaa notifications@github.com:

Thank you very much sir,sir i have code can you find what is bug in the code please,it works only to MOG2

On Wednesday, August 23, 2017 9:16 AM, Sergey Nuzhny < notifications@github.com> wrote:

Sir,I have a code of tracking i attached that file.it worked only to MOG2,could you please look at that code and tell me what modification needs to be required. Hi! I don't know what do you want. How can I advise you? This project - Multitarget-tracker contains full stack of algorithms for motion detection. I can tell you how it works.

You need CMake gui application, start it.

Select folder Mulitarget-tracker (with CMakeLists.txt file).

Select subfolder folder build.

Push a button Configure, select your MSVC compler. Wait.

If OpenCV wasn't founded automatic then select OpenCV_DIR manually. Push a button Configure again.

Push a button Generate.

Now the folder build contains a solution file (.sln). Lets open it in Visual studio! The end. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

include "opencv2/imgcodecs.hpp"

include "opencv2/imgproc.hpp"

include "opencv2/videoio.hpp"

include <opencv2/highgui.hpp>

include <opencv2/video.hpp>

include

include

include

include

include "TrackedObject.h"

using namespace cv; using namespace std;

// Global constant

const bool SHOW_ALL_OPERATION_RESULTS = true; // defines if different operation results should be shown const bool WINDOW_NAMES_READABLE = false; // if set, creates y-offset to make window names readable const int ELEMENTS_PER_ROW = 3; // defines how much windows are shown within a row const bool TRACKING_RESULTS_IN_ORIGINAL = false; // shows tracking results in original frame

// video related constants const int VIDEO_NUMBER = 1; // 1:10, choose different number for different video const bool LOOP_VIDEO = true; // defines if video should be looped infinitely

const char* VIDEO_DIRECTORY_PATH = ".\detect.avi\"; // define path to video directory

const bool SOURCE2GRAY = false; // defines, if source frame should be converted to gray

// foreground segmentation postprocessing related constants const int HISTORY = 1000; // MOG2 parameter: amount of considered frames const double VAR_THRESHOLD = 20; // MOG2 parameter: threshold for degree of minimum variance const bool DETECT_SHADOWS = false; // MOG2 parameter: defines if shadows should be detected

int thresholdMask = 100; // defines start value of threshold for // binarization of foreground frame const int THRESHOLD_MASK_MAX = 255; // defines maximum value for threshold

// morphing related constants const int KERNEL_TYPE = MORPH_ELLIPSE; // choose: MORPH_RECT, MORPH_CROSS, MORPH_ELLIPSE int kernelSizeClose = 5; // defines start value for kernel size of close op. int kernelSizeErode = 5; // defines start value for kernel size of erode op. int kernelSizeDilate = 5; // defines start value for kernel size of dilate op. const int KERNEL_SIZE_MAX = 20; // defines maximum value for threshold

// drawing related constants const Scalar DEFAULT_DRAWING_COLOR = Scalar(0, 255, 0); // defines default color of bounding boxes and labels const int CONTOUR_MIN_AREA = 500; // defines minimum area of contours const int BBOX_MIN_AREA = 3000; // defines minimum area of bounding boxes const bool SHOW_LABELS = false; // shows labels of object adjusted to the bounding box const int HEATMAP_RANGE = 25; // defines size of different colors in heat map // and threshold (color of highest weight)

// bounding box and label related constants const float ALPHA = 0.5; // defines opacity of rectangle and label const int LABEL_HEIGHT = 20; // defines height of label const char* TEST_NAME = "Car"; // defines name for test labels

const int TEXT_FONT_FACE = 2; // defines font of label text const int TEXT_THICKNESS = 2; // defines thickness of label text const int TEXT_LINE_TYPE = 0; // defines line type of label text const double TEXT_FONT_SCALE = 0.7; // defines scale of label text

// tracking related constants const bool TRACKING = true; // defines if tracking is active const bool STATEFUL_TRACKING = true; // defines if stateful tracking with TrackedObjects is active const int MAX_COUNT_TOTAL = 150; // defines max no. of tracked points in total const int MAX_COUNT_PER_CAR = 25; // defines max no. of tracked points per car const int MIN_POINTS_TRACKED_OBJECT = 3; // defines min no. of points for tracked objects const int INIT_INTERVAL = 15; // defines, after how many frames the tracker should initialize new points const TermCriteria termcrit(TermCriteria::COUNT | TermCriteria::EPS, 20, 0.03); // defines criteria for klt tracker const Size subPixWinSize(10, 10); const Size winSize(31, 31);

/// Global variables

// window related variables int scale = 10; // scales windows (and mats) after processing before displaying // (10 is equal to standard size, 20 to doubled size)

// video related variables int frameStart = 1; // defines frame to start (set in video chooser) int frameEnd = 0; // defines frame to end (set in video chooser)

// frame related variables

Mat frame; //current frame Mat fgMaskMOG2; //fg mask fg mask generated by MOG2 method Mat frameGray; // current frame in grayscale Mat frameGrayPrev; // previous frame in grayscale Mat frameThreshold; // frame with applied threshold (to elimante shadows etc.) Mat frameMorphed; // morphed frame (to elimante noise) Mat frameMasked; // morphed frame applied as a mask to the original frame Mat frameContours; // drawn contours Mat frameKltTracking; // klt tracking lanes

Ptr pMOG2; //MOG2 Background subtractor

// tracking related variables bool resetFeatures = false; Point2f point; vector points[2]; int amountHandledContoursPrev = 0;

vector trackedObjects; int objectCounter;

int keyboard; //input from keyboard

/// Function Headers

char* videoChooser();

void processVideo(char* videoFilename);

void showFrame(char* windowName, Mat frame, int width,int height, int position);

void resizeFrames();

void drawBoundingBoxes(vector<vector> contours, Mat frame);

void showControlPanel();

void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector> contours);

void trackObjects(vector<vector> contours, size_t currentPoint);

size_t* getCorrespondingObject(vector contour);

void handleProcessedPoints();

Scalar getHeatMapColor(float value);

void processVideo(char videoFilename); void processImages(char firstFrameFilename);

char* VIDEO_FILE = "visiontraffic.avi";

int main(int argc, char* argv[]) {

char* videoPath = new char[100];

/// initialize variables // define path of video file strcpy(videoPath, VIDEO_DIRECTORY_PATH); strcat(videoPath, videoChooser());

//create GUI windows namedWindow("Original"); namedWindow("Foreground (MOG2)"); namedWindow("Threshold"); namedWindow("Morphed"); namedWindow("Contours"); namedWindow("Masked Original Frame"); namedWindow("KLT Tracking");

//create Background Subtractor objects pMOG2 = createBackgroundSubtractorMOG2(HISTORY, VAR_THRESHOLD, DETECT_SHADOWS); //MOG2 approach

//input data coming from a video processVideo(VIDEO_FILE); //destroy GUI windows destroyAllWindows(); return EXIT_SUCCESS; }

void processVideo(char* videoFilename) {

//create the capture object VideoCapture capture(videoFilename); if (!capture.isOpened()) { //error in opening the video input cerr << "Unable to open video file: " << videoFilename << endl; exit(EXIT_FAILURE); }

///read input data. ESC or 'q' for quitting while ((char)keyboard != 'q' && (char)keyboard != 27) { //read the current frame if (!capture.read(frame)) { cerr << "Unable to read next frame." << endl; cerr << "Exiting..." << endl; exit(EXIT_FAILURE); }

//update the background model pMOG2->apply(frame, fgMaskMOG2);

stringstream ss; rectangle(frame, cv::Point(10, 2), cv::Point(100, 20), cv::Scalar(255, 255, 255), -1); ss << capture.get(CAP_PROP_POS_FRAMES); string frameNumberString = ss.str(); putText(frame, frameNumberString.c_str(), cv::Point(15, 15), FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0, 0));

imshow("Frame", frame); imshow("FG Mask MOG 2", fgMaskMOG2);

pMOG2->apply(frame, fgMaskMOG2);

imshow("Frame", frame); imshow("FG Mask MOG 2", fgMaskMOG2);

//get the frame number and write it on the current frame rectangle(frame, Point(10, 2), Point(100, 20), Scalar(255, 255, 255), -1); frameNumberCurrent << capture.get(CAP_PROP_POS_FRAMES); frameInfo << frameNumberCurrent.str() << "/" << frameNumberTotal.str(); putText(frame, frameInfo.str(), Point(15, 15), FONT_HERSHEY_SIMPLEX, 0.5, Scalar(0, 0, 0));

//get the input from the keyboard keyboard = waitKey(30);

/// variable declaration and initialization // morphing related variables (initialized during each loop, since parameters are changable) Mat kernelClose; Mat kernelErode; Mat kernelDilate;

// frame related variables stringstream frameNumberTotal; stringstream frameNumberCurrent; stringstream frameInfo; int frameWidth = 0, frameHeight = 0;

// contour related variables vector<vector> contours; vector hierarchy;

// displaying related variables int r = 1; // counter for row

//// video processing

frameNumberTotal << capture.get(CAP_PROP_FRAME_COUNT); frameWidth = (int)capture.get(CV_CAP_PROP_FRAME_WIDTH); frameHeight = (int)capture.get(CV_CAP_PROP_FRAME_HEIGHT);

if (frameEnd == 0) frameEnd = (int)capture.get(CAP_PROP_FRAME_COUNT);

// set video to defined start frame capture.set(CAP_PROP_POS_FRAMES, frameStart);

// show control panel (enables modification of some variables in realtime) showControlPanel();

while ((char)keyboard != 27) { // read the current frame if (!capture.read(frame)) { cerr << "Unable to read next frame." << endl; cerr << "Exiting..." << endl; exit(EXIT_FAILURE); }

// convert original frame to grayscale if specified by constant if (SOURCE2GRAY) frameGray.copyTo(frame);

// re-initialize objects with changable parameters if (kernelSizeClose > 0) kernelClose = getStructuringElement(KERNEL_TYPE, Size(kernelSizeClose, kernelSizeClose)); if (kernelSizeErode > 0) kernelErode = getStructuringElement(KERNEL_TYPE, Size(kernelSizeErode, kernelSizeErode)); if (kernelSizeDilate > 0) kernelDilate = getStructuringElement(KERNEL_TYPE, Size(kernelSizeDilate, kernelSizeDilate));

} // apply threshold to MOG2 frame to get rid of shadows etc threshold(fgMaskMOG2, frameThreshold, thresholdMask, 255, THRESH_BINARY);

// apply morphology operations to thresholded frame

if (kernelSizeClose > 0) morphologyEx(frameMorphed, frameMorphed, MORPH_CLOSE, kernelClose); if (kernelSizeErode > 0) morphologyEx(frameMorphed, frameMorphed, MORPH_ERODE, kernelErode); if (kernelSizeDilate > 0) morphologyEx(frameMorphed, frameMorphed, MORPH_DILATE, kernelDilate);

frameThreshold.copyTo(frameMorphed);

// save previous masked frame as grayscale frame for later tracking if (!frameMasked.empty()) cvtColor(frameMasked, frameGrayPrev, CV_RGB2GRAY);

// apply morphed mask, but save before the morphed frame before frameMorphed.copyTo(frameContours); frameMorphed.copyTo(frameMasked); frame.copyTo(frameMasked, frameMasked);

// save new masked frame as grayscale frame for later tracking, also save mask to KltTracking frame cvtColor(frameMasked, frameGray, CV_RGB2GRAY); frameMasked.copyTo(frameKltTracking);

// draw and fill contours findContours(frameContours, contours, hierarchy, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE); setIdentity(frameContours, Scalar(0, 0, 0)); drawContours(frameContours, contours, -1, Scalar(255, 255, 255), -1);

// draws rectangle around identified contours drawBoundingBoxes(contours, frameMasked);

// prepare for and apply tracking algorithm (use grayscale copies of masked frame) if (!frameGrayPrev.empty() && TRACKING) applyKltTracking(frameGray, frameGrayPrev, frameKltTracking, contours);

/// show the different frames (from last to first row, to overlay windows bar) resizeFrames(); // resize all frames

if (SHOW_ALL_OPERATION_RESULTS) { // row 3 r = 3; showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);

// row 2 r = 2; showFrame("Morphed", frameMorphed, frameWidth, frameHeight, 1 + (r - 1)ELEMENTS_PER_ROW); showFrame("Contours", frameContours, frameWidth, frameHeight, 2 + (r - 1)ELEMENTS_PER_ROW); showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 3

(r - 1)*ELEMENTS_PER_ROW);

// row 1 r = 1; showFrame("Original", frame, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW); showFrame("Foreground (MOG2)", fgMaskMOG2, frameWidth, frameHeight, 2 + (r

1)ELEMENTS_PER_ROW); showFrame("Threshold", frameThreshold, frameWidth, frameHeight, 3 + (r - 1)ELEMENTS_PER_ROW);

} else { // row 1 r = 1; showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 1

(r - 1)ELEMENTS_PER_ROW); showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 2 + (r - 1)ELEMENTS_PER_ROW); }

// shows image for the specified amount of milliseconds keyboard = waitKey(30);

switch (keyboard) { case 'r': resetFeatures = true; // reset on pressed 'r' break; }

// loop through the video if end is reached and loop is set if (LOOP_VIDEO && (frameNumberCurrent.str().compare(std::to_string(frameEnd)) == 0)) capture.set(CAP_PROP_POS_FRAMES, frameStart);

// ensure that new feature points are initialized every INIT_INTERVAL frames for the tracker if (std::stoi(frameNumberCurrent.str()) % INIT_INTERVAL == 0 && trackedObjects.size() > 0) resetFeatures = true;

// cleaning operation: clean string streams frameNumberCurrent.str(""); frameInfo.str(""); }

// cleaning operation: release capture object capture.release();

//imshow("FG Mask MOG 2", fgMaskMOG2);

//get the input from the keyboard //keyboard = waitKey(30); //} //delete capture object ////capture.release(); }

char* videoChooser() {

// variable declaration char* videoFile = "dog.avi"; // saves name of selected video file

// chooses video file depending on chosen video number switch (VIDEO_NUMBER) { case 1: videoFile = "visiontraffic.avi"; frameStart = 90; frameEnd = 440; break;

case 2: videoFile = "viptraffic.avi"; break;

case 3: videoFile = "highwayII_raw.avi"; break;

case 4: videoFile = "dispersed.ogv"; break;

case 5: videoFile = "highspeed.ogv"; break;

case 6: videoFile = "hightraffic-dark.ogv"; break;

case 7: videoFile = "hightraffic-occlude.ogv"; break;

case 8: videoFile = "threecars.ogv"; break;

case 9: videoFile = "turntraffic-dark.ogv"; break;

case 10: videoFile = "visor_1261565143617_cesta1.wmv"; break;

default: break; }

return videoFile; }

void showFrame(char* windowName, Mat frame, int width, int height, int position) { // variable declaration and initialization int x = 0, // x-coordinates for position y = 0, // y-coordinates for position yOffset = 30; // offset for y-coordinates, since windows bar does not count into frame height // strategy for y-offset: create windows from last row to first to overlay the bar

// create y-Offset if defince to make titles of windows readable if (!WINDOW_NAMES_READABLE) { yOffset += -32; } else if (position > ELEMENTS_PER_ROW) { yOffset += 32; }

// create window and display defined frame

cv::namedWindow(windowName); cv::imshow(windowName, frame);

// resize stats for position calculation, if defined if (scale != 10) { // filter for the case, that it's zero, since range is not adjustable in opencv gui if (scale == 0) scale = 1;

// calculate scalefactor from value of scale variable float scaleFactor = (float)scale / 10;

// calculate new width and height width = (int)(width scaleFactor); height = (int)(height scaleFactor); }

//// get positioning

// get x coordinates with width if (((position - 1) % ELEMENTS_PER_ROW) != 0) x = ((position - 1) % ELEMENTS_PER_ROW) * width;

// get y coordinates with height if (position > ELEMENTS_PER_ROW) y = (position - (position - 1) % ELEMENTS_PER_ROW) / ELEMENTS_PER_ROW * height;

// place window at calculated position moveWindow(windowName, x, y + yOffset); }

void resizeFrames() { //resize all frames, if defined if (scale != 10 && scale > 0) { // filter for the case, that it's zero, since range is not adjustable in opencv gui if (scale == 0) scale = 1;

// calculate scalefactor from value of scale variable float scaleFactor = (float)scale / 10;

// resize all frames resize(frame, frame, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(fgMaskMOG2, fgMaskMOG2, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameThreshold, frameThreshold, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameMorphed, frameMorphed, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameMasked, frameMasked, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameContours, frameContours, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameKltTracking, frameKltTracking, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); } }

void drawBoundingBoxes(vector<vector> contours, Mat frame) { // variable declaration int amountHandledContours = 0; Scalar drawingColor = DEFAULT_DRAWING_COLOR;

// create bounding rect of object and draw on defined frame for (vector<vector>::iterator itc = contours.begin(); itc != contours.end(); itc++) { // variable declaration and initialization Rect boundingBox = boundingRect(Mat(*itc)); Mat overlay;

// only handle rectangles with a minimum area if (boundingBox.area() >= CONTOUR_MIN_AREA) { if (contourArea(*itc) >= CONTOUR_MIN_AREA) {

// get corresponding object, if there's one, to define the drawing color size_t indexCorrespondingObject = getCorrespondingObject(itc); if (indexCorrespondingObject != NULL) drawingColor = trackedObjects[*indexCorrespondingObject].getColor();

// draw contours regardless of area drawContours(frame, contours, -1, drawingColor, 1);

// handle bounding boxes (but only if they're bigger than defined minimum area) if (boundingBox.area() >= BBOX_MIN_AREA) { // copy frame to frame.copyTo(overlay);

// draw rectangle rectangle(overlay, boundingBox, drawingColor, 3);

// draw test label if constant is set if (SHOW_LABELS) rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y), Point(boundingBox.x + boundingBox.width + 1, boundingBox.y - LABEL_HEIGHT), drawingColor, -1);

// combine transparent overlay of bounding rect with original frame cv::addWeighted(overlay, ALPHA, frame, 1 - ALPHA, 0, frame);

// afterwards, put text with object name into the label, if constant is set if (SHOW_LABELS) putText(frame, TEST_NAME, Point(boundingBox.x + 10, boundingBox.y - 2), TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(0, 0, 0), TEXT_THICKNESS, TEXT_LINE_TYPE, false);

// count handled contours amountHandledContours++; } } } }

// re-initialize feature tracker if new objects appeared in this frame if (amountHandledContours > amountHandledContoursPrev) { resetFeatures = true; cout << amountHandledContours << "/" << amountHandledContoursPrev << endl; }

amountHandledContoursPrev = amountHandledContours; }

void showControlPanel() { // constant declaration const int WIDTH = 400, HEIGHT = 240;

const char nameWindow = "Control Panel"; const char nameTrackbarWindowScale = "Scale"; const char nameTrackbarThreshold = "Threshold"; const char nameTrackbarKernelSizeClose = "SizeClose"; const char nameTrackbarKernelSizeErode = "SizeErode"; const char nameTrackbarKernelSizeDilate = "SizeDilate";

// create window namedWindow(nameWindow, CV_WINDOW_AUTOSIZE | CV_GUI_NORMAL); resizeWindow(nameWindow, WIDTH, HEIGHT);

// window relating parameters createTrackbar(nameTrackbarWindowScale, nameWindow, &scale, 20, NULL); createTrackbar(nameTrackbarThreshold, nameWindow, &thresholdMask, THRESHOLD_MASK_MAX, NULL);

createTrackbar(nameTrackbarKernelSizeClose, nameWindow, &kernelSizeClose, KERNEL_SIZE_MAX, NULL); createTrackbar(nameTrackbarKernelSizeErode, nameWindow, &kernelSizeErode, KERNEL_SIZE_MAX, NULL); createTrackbar(nameTrackbarKernelSizeDilate, nameWindow, &kernelSizeDilate, KERNEL_SIZE_MAX, NULL); }

/**

@function for klt tracking */ void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector> contours) {

/// variable declaration

vector newPoints;

/// calculate features and then the optical flow if (resetFeatures) { // automatic initialization goodFeaturesToTrack(frameGray, newPoints, MAX_COUNT_TOTAL, 0.01, 10, Mat(), 3, 0, 0.04); cornerSubPix(frameGray, newPoints, subPixWinSize, Size(-1, -1), termcrit);

//points[0].clear(); // could solve performance issue, but creates strange behaviour: // trackedObjects also reset every reset //points[1].clear();

// initialize with all available points from tracked objects for (std::vector::size_type l = 0; l != trackedObjects.size(); l++) for (std::vector::size_type j = 0; j != trackedObjects[l].getPoints().size(); j++) { points[0].push_back(trackedObjects[l].getPoints()[j]); } } if (!points[0].empty() || trackedObjects.size() > 0) { // variable declaration vector status; vector err; size_t i, k;

// check if previous frame is existing, otherwise copy the current one if (frameGrayPrev.empty()) frameGray.copyTo(frameGrayPrev);

// calculate the optical flow with KLT calcOpticalFlowPyrLK(frameGrayPrev, frameGray, points[0], points[1], status, err, winSize, 3, termcrit, 0, 0.001);

/// iterate through all found points for (i = k = 0; i < points[1].size(); i++) {

// don't handle point, if there's no corresponding point from the last frame if (!status[i]) continue;

// otherwise save and draw point (shrink's point vector, resizes after the for loop) points[1][k++] = points[1][i];

/// get corresponding contour and object, otherwise create new tracked object if (STATEFUL_TRACKING) trackObjects(contours, i); else circle(frameKltTracking, points[1][i], 6, DEFAULT_DRAWING_COLOR, 2, 8); }

points[1].resize(k); }

// handle the processed points handleProcessedPoints();

// save that initialization was done and not needed anymore, also add new points if (resetFeatures) { resetFeatures = false;

for (std::vector::size_type l = 0; l != newPoints.size(); l++) points[1].push_back(newPoints[l]); }

// save the current values as the previous ones if (!resetFeatures) std::swap(points[1], points[0]);

cv::swap(frameGrayPrev, frameGrayPrev); }

void trackObjects(vector<vector> contours, size_t currentPoint) { /// variable declaration int containedInContour; bool foundObject = false; bool foundContour = false; bool foundObjectOfContour = false; vector::size_type m; // iterator for contours size_t* correspondingObject = NULL; // pointer for corresponding object

/// get contour of point for (m = 0; m != contours.size(); m++) { // checks if point is inside or on the edge of the contour containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false);

// exit loop if corresponding contour is found if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) { foundContour = true; break; } }

if (trackedObjects.size() > 0) { // check for each point in the tracked object if it already exists for (std::vector::size_type l = 0; l != trackedObjects.size(); l++) { size_t* containedInObject = NULL;

if (trackedObjects[l].containsOldPoint(points[0][currentPoint]) != NULL) { // add the point (it will increase weight automatically instead) trackedObjects[l].addPoint(points[1][currentPoint], points[0][currentPoint]); foundObject = true;

// update contour of object, if contour was not modified this frame yet bool foundContour = false; if (!trackedObjects[l].getContourStatus()) { std::vector::size_type m; for (m = 0; m != contours.size(); m++) { // checks if point is inside or on the edge of the contour containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false);

// exit loop if corresponding contour is found if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) { foundContour = true; break; } }

// update corresponding contour, if one was found if (foundContour) trackedObjects[l].setContour(contours[m]); } }

} }

// create new object with this point if (!foundObject && foundContour) { size_t* containedInObject; containedInObject = getCorrespondingObject(contours[m]);

if (containedInObject != NULL) { // contour belongs to an existing object, so add the point size_t containedInObjectVal = *containedInObject;

// add points if (trackedObjects[containedInObjectVal].getPoints().size() < MAX_COUNT_PER_CAR) trackedObjects[containedInObjectVal].addPoint(points[1][currentPoint], points[0][currentPoint]);

foundObjectOfContour = true; }

// create new tracked object, since there's no existing one with this contour if (!foundObjectOfContour) { char name[11] = "Object_"; char objectNumber[4]; sprintf(objectNumber, "%d", (int)(objectCounter + 1)); strcat(name, objectNumber);

TrackedObject* trObj = new TrackedObject(name); trObj->addPoint(points[1][currentPoint], points[0][currentPoint]);

trObj->setContour(contours[m]);

trackedObjects.push_back(*trObj); objectCounter++; } } }

size_t getCorrespondingObject(vector contour) { // variable declaration size_t position = NULL;

// find out if contour is assigned to existing object if (trackedObjects.size() > 0 && contour.size() > 0) { for (std::vector::size_type i = 0; i != trackedObjects.size(); i++) { if (trackedObjects[i].getContour().size() == contour.size()) { double comparisonValue; comparisonValue = matchShapes(trackedObjects[i].getContour(), contour, CV_CONTOURS_MATCH_I1, NULL); if (comparisonValue == 0) { position = &i; break; } } } }

return position; }

void handleProcessedPoints() { // get original frame, if constant defines, that results should be shown in original if (TRACKING_RESULTS_IN_ORIGINAL) frame.copyTo(frameKltTracking);

if (trackedObjects.size() > 0) { for (std::vector::size_type i = 0; i != trackedObjects.size(); i++) {

// process all points according to the status trackedObjects[i].processStatus();

if (trackedObjects[i].getPoints().size() <= MIN_POINTS_TRACKED_OBJECT) { trackedObjects.erase(trackedObjects.begin() + i); i--; continue; }

/// draw bounding box Rect boundingBox = boundingRect(trackedObjects[i].getContour()); Mat overlay; frameKltTracking.copyTo(overlay);

/// draw rectangle and label rectangle(overlay, boundingBox, trackedObjects[i].getColor(), 3); rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y), Point(boundingBox.x + boundingBox.width + 1, boundingBox.y - LABEL_HEIGHT), trackedObjects[i].getColor(), -1);

// combine transparent overlay of bounding rect with original frame cv::addWeighted(overlay, ALPHA, frameKltTracking, 1 - ALPHA, 0, frameKltTracking);

// afterwards, put text with object name into the label, if constant is set putText(frameKltTracking, trackedObjects[i].getName(), Point(boundingBox.x

10, boundingBox.y - 2), TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(200, 200, 200), TEXT_THICKNESS, TEXT_LINE_TYPE, false);

/// draw all feature points in the color of the corresponding object for (std::vector::size_type j = 0; j != trackedObjects[i].getPoints().size(); j++) { // handle weight of point int w = trackedObjects[i].getWeight()[j]; int size = (int)trackedObjects[i].getPoints().size();

if (w != -1) { // point not found // draw filled circle for new points if (w == 1) w = -1; else if (w >= HEATMAP_RANGE) w = HEATMAP_RANGE;

if (w != -1) circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, getHeatMapColor(1 - (float)w / (float)HEATMAP_RANGE), -1, 8); circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, trackedObjects[i].getColor(), 2, 8); } } } } }

Scalar getHeatMapColor(float value) { float red, green, blue; const int NUM_COLORS = 4; static float color[NUM_COLORS][3] = { { 0, 0, 1 }, { 0, 1, 0 }, { 1, 1, 0 }, { 1, 0, 0 } }; // A static array of 4 colors: (blue, green, yellow, red) using {r,g,b} for each.

int idx1; // |-- Our desired color will be between these two indexes in "color". int idx2; // | float fractBetween = 0; // Fraction between "idx1" and "idx2" where our value is.

if (value <= 0) { idx1 = idx2 = 0; } // accounts for an input <=0 else if (value >= 1) { idx1 = idx2 = NUM_COLORS - 1; } // accounts for an input >=0 else { value = value * (NUM_COLORS - 1); idx1 = (int)floor(value); // Our desired color will be after this index. idx2 = idx1 + 1; // ... and before this index (inclusive). fractBetween = value - float(idx1); // Distance between the two indexes (0-1). }

red = ((color[idx2][0] - color[idx1][0])fractBetween + color[idx1][0]) 255; green = ((color[idx2][1] - color[idx1][1])*fractBetween + color[idx1][1])

255; blue = ((color[idx2][2] - color[idx1][2])fractBetween + color[idx1][2]) 255;

return Scalar(red, green, blue);

}

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Smorodov/Multitarget-tracker/issues/43#issuecomment-324215822, or mute the thread https://github.com/notifications/unsubscribe-auth/ABu3jdDAJ3O_HP4btBfv1NveFOQHM5plks5sa6YhgaJpZM4O-EOi .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

No I have no such videos.

Smorodov / Multitarget-tracker

Tracking #43

include "opencv2/imgcodecs.hpp"

include "opencv2/imgproc.hpp"

include "opencv2/videoio.hpp"

include <opencv2/highgui.hpp>

include <opencv2/video.hpp>

include

include

include

include

include "TrackedObject.h"

include "opencv2/imgcodecs.hpp"

include "opencv2/imgproc.hpp"

include "opencv2/videoio.hpp"

include <opencv2/highgui.hpp>

include <opencv2/video.hpp>

include

include

include

include

include "TrackedObject.h"

include "opencv2/imgcodecs.hpp"

include "opencv2/imgproc.hpp"

include "opencv2/videoio.hpp"

include <opencv2/highgui.hpp>

include <opencv2/video.hpp>

include

include

include

include

include "TrackedObject.h"

include "opencv2/imgcodecs.hpp"

include "opencv2/imgproc.hpp"

include "opencv2/videoio.hpp"

include <opencv2/highgui.hpp>

include <opencv2/video.hpp>

include

include

include

include

include "TrackedObject.h"