Closed dulithaaaa closed 5 years ago
Hi! Do you work with Windows and MS VC? In any case you need a CMake and configure project with it.
Dear Sir,I worked with visual studio 2010 and opencv 2.4.9. Sir,other problem is,all the header files and source files should be added or not,i want to tracking moving people video
Sir,After install Cmake then waht to do
can you step by step how to compile project. I installed the CMake and also set the path.
I have bad news for you. This project requires OpenCV 3.x and compiler with C++11 standart support. Visual studio 2010 and opencv 2.4.9 are not supported - too old versions.
Dear Sir,I installd visual studio 2015 and opencv 3.1.Then what shall i do sir,explain me sir,I am a beginner for this subject.Your kindly support i will hope
On Tuesday, August 22, 2017 1:28 PM, Sergey Nuzhny <notifications@github.com> wrote:
I have bad news for you. This project requires OpenCV 3.x and compiler with C++11 standart support. Visual studio 2010 and opencv 2.4.9 are not supported - too old versions.— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.
Sir,I have a code of tracking i attached that file.it worked only to MOG2,could you please look at that code and tell me what modification needs to be required.
On Wednesday, August 23, 2017 6:54 AM, Dulitha Sadhanuwan <dulithasadhanuwan@yahoo.com> wrote:
Dear Sir,I installd visual studio 2015 and opencv 3.1.Then what shall i do sir,explain me sir,I am a beginner for this subject.Your kindly support i will hope
On Tuesday, August 22, 2017 1:28 PM, Sergey Nuzhny <notifications@github.com> wrote:
I have bad news for you. This project requires OpenCV 3.x and compiler with C++11 standart support. Visual studio 2010 and opencv 2.4.9 are not supported - too old versions.— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.
using namespace cv; using namespace std;
// Global constant
const bool SHOW_ALL_OPERATION_RESULTS = true; // defines if different operation results should be shown const bool WINDOW_NAMES_READABLE = false; // if set, creates y-offset to make window names readable const int ELEMENTS_PER_ROW = 3; // defines how much windows are shown within a row const bool TRACKING_RESULTS_IN_ORIGINAL = false; // shows tracking results in original frame
// video related constants const int VIDEO_NUMBER = 1; // 1:10, choose different number for different video const bool LOOP_VIDEO = true; // defines if video should be looped infinitely
const char* VIDEO_DIRECTORY_PATH = ".\detect.avi\"; // define path to video directory
const bool SOURCE2GRAY = false; // defines, if source frame should be converted to gray
// foreground segmentation postprocessing related constants const int HISTORY = 1000; // MOG2 parameter: amount of considered frames const double VAR_THRESHOLD = 20; // MOG2 parameter: threshold for degree of minimum variance const bool DETECT_SHADOWS = false; // MOG2 parameter: defines if shadows should be detected
int thresholdMask = 100; // defines start value of threshold for // binarization of foreground frame const int THRESHOLD_MASK_MAX = 255; // defines maximum value for threshold
// morphing related constants const int KERNEL_TYPE = MORPH_ELLIPSE; // choose: MORPH_RECT, MORPH_CROSS, MORPH_ELLIPSE int kernelSizeClose = 5; // defines start value for kernel size of close op. int kernelSizeErode = 5; // defines start value for kernel size of erode op. int kernelSizeDilate = 5; // defines start value for kernel size of dilate op. const int KERNEL_SIZE_MAX = 20; // defines maximum value for threshold
// drawing related constants const Scalar DEFAULT_DRAWING_COLOR = Scalar(0, 255, 0); // defines default color of bounding boxes and labels const int CONTOUR_MIN_AREA = 500; // defines minimum area of contours const int BBOX_MIN_AREA = 3000; // defines minimum area of bounding boxes const bool SHOW_LABELS = false; // shows labels of object adjusted to the bounding box const int HEATMAP_RANGE = 25; // defines size of different colors in heat map // and threshold (color of highest weight)
// bounding box and label related constants const float ALPHA = 0.5; // defines opacity of rectangle and label const int LABEL_HEIGHT = 20; // defines height of label const char* TEST_NAME = "Car"; // defines name for test labels
const int TEXT_FONT_FACE = 2; // defines font of label text const int TEXT_THICKNESS = 2; // defines thickness of label text const int TEXT_LINE_TYPE = 0; // defines line type of label text const double TEXT_FONT_SCALE = 0.7; // defines scale of label text
// tracking related constants const bool TRACKING = true; // defines if tracking is active const bool STATEFUL_TRACKING = true; // defines if stateful tracking with TrackedObjects is active const int MAX_COUNT_TOTAL = 150; // defines max no. of tracked points in total const int MAX_COUNT_PER_CAR = 25; // defines max no. of tracked points per car const int MIN_POINTS_TRACKED_OBJECT = 3; // defines min no. of points for tracked objects const int INIT_INTERVAL = 15; // defines, after how many frames the tracker should initialize new points const TermCriteria termcrit(TermCriteria::COUNT | TermCriteria::EPS, 20, 0.03); // defines criteria for klt tracker const Size subPixWinSize(10, 10); const Size winSize(31, 31);
/// Global variables
// window related variables int scale = 10; // scales windows (and mats) after processing before displaying // (10 is equal to standard size, 20 to doubled size)
// video related variables int frameStart = 1; // defines frame to start (set in video chooser) int frameEnd = 0; // defines frame to end (set in video chooser)
// frame related variables
Mat frame; //current frame Mat fgMaskMOG2; //fg mask fg mask generated by MOG2 method Mat frameGray; // current frame in grayscale Mat frameGrayPrev; // previous frame in grayscale Mat frameThreshold; // frame with applied threshold (to elimante shadows etc.) Mat frameMorphed; // morphed frame (to elimante noise) Mat frameMasked; // morphed frame applied as a mask to the original frame Mat frameContours; // drawn contours Mat frameKltTracking; // klt tracking lanes
Ptr
// tracking related variables
bool resetFeatures = false;
Point2f point;
vector
vector
int keyboard; //input from keyboard
/// Function Headers
char* videoChooser();
void processVideo(char* videoFilename);
//void threshold_demo(int,void*);
void showFrame(char* windowName, Mat frame, int width,int height, int position);
void resizeFrames();
void drawBoundingBoxes(vector<vector
void showControlPanel();
void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector
void trackObjects(vector<vector
size_t* getCorrespondingObject(vector
void handleProcessedPoints();
Scalar getHeatMapColor(float value);
void processVideo(char videoFilename); void processImages(char firstFrameFilename);
char* VIDEO_FILE = "visiontraffic.avi";
int main(int argc, char* argv[]) {
char* videoPath = new char[100];
/// initialize variables
// define path of video file
strcpy(videoPath, VIDEO_DIRECTORY_PATH);
strcat(videoPath, videoChooser());
//create GUI windows
namedWindow("Original");
namedWindow("Foreground (MOG2)");
namedWindow("Threshold");
namedWindow("Morphed");
namedWindow("Contours");
namedWindow("Masked Original Frame");
namedWindow("KLT Tracking");
//create Background Subtractor objects
pMOG2 = createBackgroundSubtractorMOG2(HISTORY, VAR_THRESHOLD, DETECT_SHADOWS); //MOG2 approach
//input data coming from a video
processVideo(VIDEO_FILE);
//destroy GUI windows
destroyAllWindows();
return EXIT_SUCCESS;
}
void processVideo(char* videoFilename) {
/// variable declaration and initialization
// morphing related variables (initialized during each loop, since parameters are changable)
Mat kernelClose;
Mat kernelErode;
Mat kernelDilate;
// frame related variables
stringstream frameNumberTotal;
stringstream frameNumberCurrent;
stringstream frameInfo;
int frameWidth = 0,
frameHeight = 0;
// contour related variables
vector<vector<Point>> contours;
vector<Vec4i> hierarchy;
// displaying related variables
int r = 1; // counter for row
//// video processing
//create the capture object
VideoCapture capture(videoFilename);
if (!capture.isOpened()) {
//error in opening the video input
cerr << "Unable to open video file: " << videoFilename << endl;
exit(EXIT_FAILURE);
}
frameNumberTotal << capture.get(CAP_PROP_FRAME_COUNT);
frameWidth = (int)capture.get(CV_CAP_PROP_FRAME_WIDTH);
frameHeight = (int)capture.get(CV_CAP_PROP_FRAME_HEIGHT);
if (frameEnd == 0)
frameEnd = (int)capture.get(CAP_PROP_FRAME_COUNT);
// set video to defined start frame
capture.set(CAP_PROP_POS_FRAMES, frameStart);
// show control panel (enables modification of some variables in realtime)
showControlPanel();
while ((char)keyboard != 27) {
// read the current frame
if (!capture.read(frame)) {
cerr << "Unable to read next frame." << endl;
cerr << "Exiting..." << endl;
exit(EXIT_FAILURE);
}
// convert original frame to grayscale if specified by constant
if (SOURCE2GRAY)
frameGray.copyTo(frame);
// re-initialize objects with changable parameters
if (kernelSizeClose > 0)
kernelClose = getStructuringElement(KERNEL_TYPE, Size(kernelSizeClose, kernelSizeClose));
if (kernelSizeErode > 0)
kernelErode = getStructuringElement(KERNEL_TYPE, Size(kernelSizeErode, kernelSizeErode));
if (kernelSizeDilate > 0)
kernelDilate = getStructuringElement(KERNEL_TYPE, Size(kernelSizeDilate, kernelSizeDilate));
pMOG2->apply(frame, fgMaskMOG2);
///read input data. ESC or 'q' for quitting
while ((char)keyboard != 'q' && (char)keyboard != 27) {
//read the current frame
if (!capture.read(frame)) {
cerr << "Unable to read next frame." << endl;
cerr << "Exiting..." << endl;
exit(EXIT_FAILURE);
}
//update the background model
pMOG2->apply(frame, fgMaskMOG2);
stringstream ss;
rectangle(frame, cv::Point(10, 2), cv::Point(100, 20),
cv::Scalar(255, 255, 255), -1);
ss << capture.get(CAP_PROP_POS_FRAMES);
string frameNumberString = ss.str();
putText(frame, frameNumberString.c_str(), cv::Point(15, 15),
FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0, 0));
imshow("Frame", frame);
imshow("FG Mask MOG 2", fgMaskMOG2);
//get the input from the keyboard
keyboard = waitKey(30);
//get the frame number and write it on the current frame
rectangle(frame, Point(10, 2), Point(100, 20), Scalar(255, 255, 255), -1);
frameNumberCurrent << capture.get(CAP_PROP_POS_FRAMES);
frameInfo << frameNumberCurrent.str() << "/" << frameNumberTotal.str();
putText(frame, frameInfo.str(), Point(15, 15),
FONT_HERSHEY_SIMPLEX, 0.5, Scalar(0, 0, 0));
}
// apply threshold to MOG2 frame to get rid of shadows etc
threshold(fgMaskMOG2, frameThreshold, thresholdMask, 255, THRESH_BINARY);
// apply morphology operations to thresholded frame
if (kernelSizeClose > 0)
morphologyEx(frameMorphed, frameMorphed, MORPH_CLOSE, kernelClose);
if (kernelSizeErode > 0)
morphologyEx(frameMorphed, frameMorphed, MORPH_ERODE, kernelErode);
if (kernelSizeDilate > 0)
morphologyEx(frameMorphed, frameMorphed, MORPH_DILATE, kernelDilate);
frameThreshold.copyTo(frameMorphed);
// save previous masked frame as grayscale frame for later tracking
if (!frameMasked.empty())
cvtColor(frameMasked, frameGrayPrev, CV_RGB2GRAY);
// apply morphed mask, but save before the morphed frame before
frameMorphed.copyTo(frameContours);
frameMorphed.copyTo(frameMasked);
frame.copyTo(frameMasked, frameMasked);
// save new masked frame as grayscale frame for later tracking, also save mask to KltTracking frame
cvtColor(frameMasked, frameGray, CV_RGB2GRAY);
frameMasked.copyTo(frameKltTracking);
// draw and fill contours
findContours(frameContours, contours, hierarchy, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
setIdentity(frameContours, Scalar(0, 0, 0));
drawContours(frameContours, contours, -1, Scalar(255, 255, 255), -1);
// draws rectangle around identified contours
drawBoundingBoxes(contours, frameMasked);
// prepare for and apply tracking algorithm (use grayscale copies of masked frame)
if (!frameGrayPrev.empty() && TRACKING)
applyKltTracking(frameGray, frameGrayPrev, frameKltTracking, contours);
/// show the different frames (from last to first row, to overlay windows bar)
resizeFrames(); // resize all frames
if (SHOW_ALL_OPERATION_RESULTS) {
// row 3
r = 3;
showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
// row 2
r = 2;
showFrame("Morphed", frameMorphed, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
showFrame("Contours", frameContours, frameWidth, frameHeight, 2 + (r - 1)*ELEMENTS_PER_ROW);
showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 3 + (r - 1)*ELEMENTS_PER_ROW);
// row 1
r = 1;
showFrame("Original", frame, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
showFrame("Foreground (MOG2)", fgMaskMOG2, frameWidth, frameHeight, 2 + (r - 1)*ELEMENTS_PER_ROW);
showFrame("Threshold", frameThreshold, frameWidth, frameHeight, 3 + (r - 1)*ELEMENTS_PER_ROW);
}
else {
// row 1
r = 1;
showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 2 + (r - 1)*ELEMENTS_PER_ROW);
}
// shows image for the specified amount of milliseconds
keyboard = waitKey(30);
switch (keyboard) {
case 'r':
resetFeatures = true; // reset on pressed 'r'
break;
}
// loop through the video if end is reached and loop is set
if (LOOP_VIDEO && (frameNumberCurrent.str().compare(std::to_string(frameEnd)) == 0))
capture.set(CAP_PROP_POS_FRAMES, frameStart);
// ensure that new feature points are initialized every INIT_INTERVAL frames for the tracker
if (std::stoi(frameNumberCurrent.str()) % INIT_INTERVAL == 0 && trackedObjects.size() > 0)
resetFeatures = true;
// cleaning operation: clean string streams
frameNumberCurrent.str("");
frameInfo.str("");
}
// cleaning operation: release capture object
capture.release();
//imshow("FG Mask MOG 2", fgMaskMOG2);
//get the input from the keyboard
//keyboard = waitKey(30);
//}
//delete capture object
////capture.release();
}
char* videoChooser() {
// variable declaration
char* videoFile = "dog.avi"; // saves name of selected video file
// chooses video file depending on chosen video number
switch (VIDEO_NUMBER) {
case 1:
videoFile = "visiontraffic.avi";
frameStart = 90;
frameEnd = 440;
break;
case 2:
videoFile = "viptraffic.avi";
break;
case 3:
videoFile = "highwayII_raw.avi";
break;
case 4:
videoFile = "dispersed.ogv";
break;
case 5:
videoFile = "highspeed.ogv";
break;
case 6:
videoFile = "hightraffic-dark.ogv";
break;
case 7:
videoFile = "hightraffic-occlude.ogv";
break;
case 8:
videoFile = "threecars.ogv";
break;
case 9:
videoFile = "turntraffic-dark.ogv";
break;
case 10:
videoFile = "visor_1261565143617_cesta1.wmv";
break;
default:
break;
}
return videoFile;
}
void showFrame(char* windowName, Mat frame, int width, int height, int position) { // variable declaration and initialization int x = 0, // x-coordinates for position y = 0, // y-coordinates for position yOffset = 30; // offset for y-coordinates, since windows bar does not count into frame height // strategy for y-offset: create windows from last row to first to overlay the bar
// create y-Offset if defince to make titles of windows readable
if (!WINDOW_NAMES_READABLE) {
yOffset += -32;
}
else if (position > ELEMENTS_PER_ROW) {
yOffset += 32;
}
// create window and display defined frame
cv::namedWindow(windowName);
cv::imshow(windowName, frame);
// resize stats for position calculation, if defined
if (scale != 10) {
// filter for the case, that it's zero, since range is not adjustable in opencv gui
if (scale == 0)
scale = 1;
// calculate scalefactor from value of scale variable
float scaleFactor = (float)scale / 10;
// calculate new width and height
width = (int)(width * scaleFactor);
height = (int)(height * scaleFactor);
}
//// get positioning
// get x coordinates with width
if (((position - 1) % ELEMENTS_PER_ROW) != 0)
x = ((position - 1) % ELEMENTS_PER_ROW) * width;
// get y coordinates with height
if (position > ELEMENTS_PER_ROW)
y = (position - (position - 1) % ELEMENTS_PER_ROW) / ELEMENTS_PER_ROW * height;
// place window at calculated position
moveWindow(windowName, x, y + yOffset);
}
void resizeFrames() { //resize all frames, if defined if (scale != 10 && scale > 0) { // filter for the case, that it's zero, since range is not adjustable in opencv gui if (scale == 0) scale = 1;
// calculate scalefactor from value of scale variable
float scaleFactor = (float)scale / 10;
// resize all frames
resize(frame, frame, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
resize(fgMaskMOG2, fgMaskMOG2, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
resize(frameThreshold, frameThreshold, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
resize(frameMorphed, frameMorphed, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
resize(frameMasked, frameMasked, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
resize(frameContours, frameContours, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
resize(frameKltTracking, frameKltTracking, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
}
}
void drawBoundingBoxes(vector<vector
// create bounding rect of object and draw on defined frame
for (vector<vector<Point>>::iterator itc = contours.begin(); itc != contours.end(); itc++) {
// variable declaration and initialization
Rect boundingBox = boundingRect(Mat(*itc));
Mat overlay;
// only handle rectangles with a minimum area
if (boundingBox.area() >= CONTOUR_MIN_AREA) {
if (contourArea(*itc) >= CONTOUR_MIN_AREA) {
// get corresponding object, if there's one, to define the drawing color
size_t* indexCorrespondingObject = getCorrespondingObject(*itc);
if (indexCorrespondingObject != NULL)
drawingColor = trackedObjects[*indexCorrespondingObject].getColor();
// draw contours regardless of area
drawContours(frame, contours, -1, drawingColor, 1);
// handle bounding boxes (but only if they're bigger than defined minimum area)
if (boundingBox.area() >= BBOX_MIN_AREA) {
// copy frame to
frame.copyTo(overlay);
// draw rectangle
rectangle(overlay, boundingBox, drawingColor, 3);
// draw test label if constant is set
if (SHOW_LABELS)
rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y),
Point(boundingBox.x + boundingBox.width + 1,
boundingBox.y - LABEL_HEIGHT), drawingColor, -1);
// combine transparent overlay of bounding rect with original frame
cv::addWeighted(overlay, ALPHA, frame, 1 - ALPHA, 0, frame);
// afterwards, put text with object name into the label, if constant is set
if (SHOW_LABELS)
putText(frame, TEST_NAME, Point(boundingBox.x + 10, boundingBox.y - 2),
TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(0, 0, 0),
TEXT_THICKNESS, TEXT_LINE_TYPE, false);
// count handled contours
amountHandledContours++;
}
}
}
}
// re-initialize feature tracker if new objects appeared in this frame
if (amountHandledContours > amountHandledContoursPrev) {
resetFeatures = true;
cout << amountHandledContours << "/" << amountHandledContoursPrev << endl;
}
amountHandledContoursPrev = amountHandledContours;
}
void showControlPanel() { // constant declaration const int WIDTH = 400, HEIGHT = 240;
const char* nameWindow = "Control Panel";
const char* nameTrackbarWindowScale = "Scale";
const char* nameTrackbarThreshold = "Threshold";
const char* nameTrackbarKernelSizeClose = "SizeClose";
const char* nameTrackbarKernelSizeErode = "SizeErode";
const char* nameTrackbarKernelSizeDilate = "SizeDilate";
// create window
namedWindow(nameWindow, CV_WINDOW_AUTOSIZE | CV_GUI_NORMAL);
resizeWindow(nameWindow, WIDTH, HEIGHT);
// window relating parameters
createTrackbar(nameTrackbarWindowScale, nameWindow, &scale, 20, NULL);
createTrackbar(nameTrackbarThreshold, nameWindow, &thresholdMask, THRESHOLD_MASK_MAX, NULL);
createTrackbar(nameTrackbarKernelSizeClose, nameWindow, &kernelSizeClose, KERNEL_SIZE_MAX, NULL);
createTrackbar(nameTrackbarKernelSizeErode, nameWindow, &kernelSizeErode, KERNEL_SIZE_MAX, NULL);
createTrackbar(nameTrackbarKernelSizeDilate, nameWindow, &kernelSizeDilate, KERNEL_SIZE_MAX, NULL);
}
/**
@function for klt tracking
*/
void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector
/// variable declaration
vector
/// calculate features and then the optical flow if (resetFeatures) { // automatic initialization goodFeaturesToTrack(frameGray, newPoints, MAX_COUNT_TOTAL, 0.01, 10, Mat(), 3, 0, 0.04); cornerSubPix(frameGray, newPoints, subPixWinSize, Size(-1, -1), termcrit);
//points[0].clear(); // could solve performance issue, but creates strange behaviour:
// trackedObjects also reset every reset
//points[1].clear();
// initialize with all available points from tracked objects
for (std::vector<TrackedObject>::size_type l = 0; l != trackedObjects.size(); l++)
for (std::vector<TrackedObject>::size_type j = 0; j != trackedObjects[l].getPoints().size(); j++) {
points[0].push_back(trackedObjects[l].getPoints()[j]);
}
}
if (!points[0].empty() || trackedObjects.size() > 0) {
// variable declaration
vector
// check if previous frame is existing, otherwise copy the current one
if (frameGrayPrev.empty())
frameGray.copyTo(frameGrayPrev);
// calculate the optical flow with KLT
calcOpticalFlowPyrLK(frameGrayPrev, frameGray, points[0], points[1], status, err, winSize,
3, termcrit, 0, 0.001);
/// iterate through all found points
for (i = k = 0; i < points[1].size(); i++) {
// don't handle point, if there's no corresponding point from the last frame
if (!status[i])
continue;
// otherwise save and draw point (shrink's point vector, resizes after the for loop)
points[1][k++] = points[1][i];
/// get corresponding contour and object, otherwise create new tracked object
if (STATEFUL_TRACKING)
trackObjects(contours, i);
else
circle(frameKltTracking, points[1][i], 6, DEFAULT_DRAWING_COLOR, 2, 8);
}
points[1].resize(k);
}
// handle the processed points handleProcessedPoints();
// save that initialization was done and not needed anymore, also add new points if (resetFeatures) { resetFeatures = false;
for (std::vector<Point2f>::size_type l = 0; l != newPoints.size(); l++)
points[1].push_back(newPoints[l]);
}
// save the current values as the previous ones if (!resetFeatures) std::swap(points[1], points[0]);
cv::swap(frameGrayPrev, frameGrayPrev); }
void trackObjects(vector<vector
/// get contour of point
for (m = 0; m != contours.size(); m++) {
// checks if point is inside or on the edge of the contour
containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false);
// exit loop if corresponding contour is found
if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) {
foundContour = true;
break;
}
}
if (trackedObjects.size() > 0) {
// check for each point in the tracked object if it already exists
for (std::vector<TrackedObject>::size_type l = 0; l != trackedObjects.size(); l++) {
size_t* containedInObject = NULL;
if (trackedObjects[l].containsOldPoint(points[0][currentPoint]) != NULL) {
// add the point (it will increase weight automatically instead)
trackedObjects[l].addPoint(points[1][currentPoint], points[0][currentPoint]);
foundObject = true;
// update contour of object, if contour was not modified this frame yet
bool foundContour = false;
if (!trackedObjects[l].getContourStatus()) {
std::vector<Point>::size_type m;
for (m = 0; m != contours.size(); m++) {
// checks if point is inside or on the edge of the contour
containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false);
// exit loop if corresponding contour is found
if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) {
foundContour = true;
break;
}
}
// update corresponding contour, if one was found
if (foundContour)
trackedObjects[l].setContour(contours[m]);
}
}
}
}
// create new object with this point
if (!foundObject && foundContour) {
size_t* containedInObject;
containedInObject = getCorrespondingObject(contours[m]);
if (containedInObject != NULL) {
// contour belongs to an existing object, so add the point
size_t containedInObjectVal = *containedInObject;
// add points
if (trackedObjects[containedInObjectVal].getPoints().size() < MAX_COUNT_PER_CAR)
trackedObjects[containedInObjectVal].addPoint(points[1][currentPoint], points[0][currentPoint]);
foundObjectOfContour = true;
}
// create new tracked object, since there's no existing one with this contour
if (!foundObjectOfContour) {
char name[11] = "Object_";
char objectNumber[4];
sprintf(objectNumber, "%d", (int)(objectCounter + 1));
strcat(name, objectNumber);
TrackedObject* trObj = new TrackedObject(name);
trObj->addPoint(points[1][currentPoint], points[0][currentPoint]);
trObj->setContour(contours[m]);
trackedObjects.push_back(*trObj);
objectCounter++;
}
}
}
size_t getCorrespondingObject(vector
// find out if contour is assigned to existing object
if (trackedObjects.size() > 0 && contour.size() > 0) {
for (std::vector<TrackedObject>::size_type i = 0; i != trackedObjects.size(); i++) {
if (trackedObjects[i].getContour().size() == contour.size()) {
double comparisonValue;
comparisonValue = matchShapes(trackedObjects[i].getContour(), contour, CV_CONTOURS_MATCH_I1, NULL);
if (comparisonValue == 0) {
position = &i;
break;
}
}
}
}
return position;
}
void handleProcessedPoints() { // get original frame, if constant defines, that results should be shown in original if (TRACKING_RESULTS_IN_ORIGINAL) frame.copyTo(frameKltTracking);
if (trackedObjects.size() > 0) {
for (std::vector<TrackedObject>::size_type i = 0; i != trackedObjects.size(); i++) {
// process all points according to the status
trackedObjects[i].processStatus();
if (trackedObjects[i].getPoints().size() <= MIN_POINTS_TRACKED_OBJECT) {
trackedObjects.erase(trackedObjects.begin() + i);
i--;
continue;
}
/// draw bounding box
Rect boundingBox = boundingRect(trackedObjects[i].getContour());
Mat overlay;
frameKltTracking.copyTo(overlay);
/// draw rectangle and label
rectangle(overlay, boundingBox, trackedObjects[i].getColor(), 3);
rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y),
Point(boundingBox.x + boundingBox.width + 1,
boundingBox.y - LABEL_HEIGHT), trackedObjects[i].getColor(), -1);
// combine transparent overlay of bounding rect with original frame
cv::addWeighted(overlay, ALPHA, frameKltTracking, 1 - ALPHA, 0, frameKltTracking);
// afterwards, put text with object name into the label, if constant is set
putText(frameKltTracking, trackedObjects[i].getName(), Point(boundingBox.x + 10, boundingBox.y - 2),
TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(200, 200, 200),
TEXT_THICKNESS, TEXT_LINE_TYPE, false);
/// draw all feature points in the color of the corresponding object
for (std::vector<Point2f>::size_type j = 0; j != trackedObjects[i].getPoints().size(); j++) {
// handle weight of point
int w = trackedObjects[i].getWeight()[j];
int size = (int)trackedObjects[i].getPoints().size();
if (w != -1) { // point not found
// draw filled circle for new points
if (w == 1)
w = -1;
else if (w >= HEATMAP_RANGE)
w = HEATMAP_RANGE;
if (w != -1)
circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, getHeatMapColor(1 - (float)w / (float)HEATMAP_RANGE), -1, 8);
circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, trackedObjects[i].getColor(), 2, 8);
}
}
}
}
}
Scalar getHeatMapColor(float value) { float red, green, blue; const int NUM_COLORS = 4; static float color[NUM_COLORS][3] = { { 0, 0, 1 }, { 0, 1, 0 }, { 1, 1, 0 }, { 1, 0, 0 } }; // A static array of 4 colors: (blue, green, yellow, red) using {r,g,b} for each.
int idx1; // |-- Our desired color will be between these two indexes in "color".
int idx2; // |
float fractBetween = 0; // Fraction between "idx1" and "idx2" where our value is.
if (value <= 0) { idx1 = idx2 = 0; } // accounts for an input <=0
else if (value >= 1) { idx1 = idx2 = NUM_COLORS - 1; } // accounts for an input >=0
else
{
value = value * (NUM_COLORS - 1);
idx1 = (int)floor(value); // Our desired color will be after this index.
idx2 = idx1 + 1; // ... and before this index (inclusive).
fractBetween = value - float(idx1); // Distance between the two indexes (0-1).
}
red = ((color[idx2][0] - color[idx1][0])*fractBetween + color[idx1][0]) * 255;
green = ((color[idx2][1] - color[idx1][1])*fractBetween + color[idx1][1]) * 255;
blue = ((color[idx2][2] - color[idx1][2])*fractBetween + color[idx1][2]) * 255;
return Scalar(red, green, blue);
}
Sir,I have a code of tracking i attached that file.it worked only to MOG2,could you please look at that code and tell me what modification needs to be required.
Hi! I don't know what do you want. How can I advise you? This project - Multitarget-tracker contains full stack of algorithms for motion detection. I can tell you how it works.
Thank you very much sir,sir i have code can you find what is bug in the code please,it works only to MOG2
On Wednesday, August 23, 2017 9:16 AM, Sergey Nuzhny <notifications@github.com> wrote:
Sir,I have a code of tracking i attached that file.it worked only to MOG2,could you please look at that code and tell me what modification needs to be required. Hi! I don't know what do you want. How can I advise you? This project - Multitarget-tracker contains full stack of algorithms for motion detection. I can tell you how it works.
using namespace cv; using namespace std;
// Global constant
const bool SHOW_ALL_OPERATION_RESULTS = true; // defines if different operation results should be shown const bool WINDOW_NAMES_READABLE = false; // if set, creates y-offset to make window names readable const int ELEMENTS_PER_ROW = 3; // defines how much windows are shown within a row const bool TRACKING_RESULTS_IN_ORIGINAL = false; // shows tracking results in original frame
// video related constants const int VIDEO_NUMBER = 1; // 1:10, choose different number for different video const bool LOOP_VIDEO = true; // defines if video should be looped infinitely
const char* VIDEO_DIRECTORY_PATH = ".\detect.avi\"; // define path to video directory
const bool SOURCE2GRAY = false; // defines, if source frame should be converted to gray
// foreground segmentation postprocessing related constants const int HISTORY = 1000; // MOG2 parameter: amount of considered frames const double VAR_THRESHOLD = 20; // MOG2 parameter: threshold for degree of minimum variance const bool DETECT_SHADOWS = false; // MOG2 parameter: defines if shadows should be detected
int thresholdMask = 100; // defines start value of threshold for // binarization of foreground frame const int THRESHOLD_MASK_MAX = 255; // defines maximum value for threshold
// morphing related constants const int KERNEL_TYPE = MORPH_ELLIPSE; // choose: MORPH_RECT, MORPH_CROSS, MORPH_ELLIPSE int kernelSizeClose = 5; // defines start value for kernel size of close op. int kernelSizeErode = 5; // defines start value for kernel size of erode op. int kernelSizeDilate = 5; // defines start value for kernel size of dilate op. const int KERNEL_SIZE_MAX = 20; // defines maximum value for threshold
// drawing related constants const Scalar DEFAULT_DRAWING_COLOR = Scalar(0, 255, 0); // defines default color of bounding boxes and labels const int CONTOUR_MIN_AREA = 500; // defines minimum area of contours const int BBOX_MIN_AREA = 3000; // defines minimum area of bounding boxes const bool SHOW_LABELS = false; // shows labels of object adjusted to the bounding box const int HEATMAP_RANGE = 25; // defines size of different colors in heat map // and threshold (color of highest weight)
// bounding box and label related constants const float ALPHA = 0.5; // defines opacity of rectangle and label const int LABEL_HEIGHT = 20; // defines height of label const char* TEST_NAME = "Car"; // defines name for test labels
const int TEXT_FONT_FACE = 2; // defines font of label text const int TEXT_THICKNESS = 2; // defines thickness of label text const int TEXT_LINE_TYPE = 0; // defines line type of label text const double TEXT_FONT_SCALE = 0.7; // defines scale of label text
// tracking related constants const bool TRACKING = true; // defines if tracking is active const bool STATEFUL_TRACKING = true; // defines if stateful tracking with TrackedObjects is active const int MAX_COUNT_TOTAL = 150; // defines max no. of tracked points in total const int MAX_COUNT_PER_CAR = 25; // defines max no. of tracked points per car const int MIN_POINTS_TRACKED_OBJECT = 3; // defines min no. of points for tracked objects const int INIT_INTERVAL = 15; // defines, after how many frames the tracker should initialize new points const TermCriteria termcrit(TermCriteria::COUNT | TermCriteria::EPS, 20, 0.03); // defines criteria for klt tracker const Size subPixWinSize(10, 10); const Size winSize(31, 31);
/// Global variables
// window related variables int scale = 10; // scales windows (and mats) after processing before displaying // (10 is equal to standard size, 20 to doubled size)
// video related variables int frameStart = 1; // defines frame to start (set in video chooser) int frameEnd = 0; // defines frame to end (set in video chooser)
// frame related variables
Mat frame; //current frame Mat fgMaskMOG2; //fg mask fg mask generated by MOG2 method Mat frameGray; // current frame in grayscale Mat frameGrayPrev; // previous frame in grayscale Mat frameThreshold; // frame with applied threshold (to elimante shadows etc.) Mat frameMorphed; // morphed frame (to elimante noise) Mat frameMasked; // morphed frame applied as a mask to the original frame Mat frameContours; // drawn contours Mat frameKltTracking; // klt tracking lanes
Ptr
// tracking related variables
bool resetFeatures = false;
Point2f point;
vector
vector
int keyboard; //input from keyboard
/// Function Headers
char* videoChooser();
void processVideo(char* videoFilename);
void showFrame(char* windowName, Mat frame, int width,int height, int position);
void resizeFrames();
void drawBoundingBoxes(vector<vector
void showControlPanel();
void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector
void trackObjects(vector<vector
size_t* getCorrespondingObject(vector
void handleProcessedPoints();
Scalar getHeatMapColor(float value);
void processVideo(char videoFilename); void processImages(char firstFrameFilename);
char* VIDEO_FILE = "visiontraffic.avi";
int main(int argc, char* argv[]) {
char* videoPath = new char[100];
/// initialize variables
// define path of video file
strcpy(videoPath, VIDEO_DIRECTORY_PATH);
strcat(videoPath, videoChooser());
//create GUI windows
namedWindow("Original");
namedWindow("Foreground (MOG2)");
namedWindow("Threshold");
namedWindow("Morphed");
namedWindow("Contours");
namedWindow("Masked Original Frame");
namedWindow("KLT Tracking");
//create Background Subtractor objects
pMOG2 = createBackgroundSubtractorMOG2(HISTORY, VAR_THRESHOLD, DETECT_SHADOWS); //MOG2 approach
//input data coming from a video
processVideo(VIDEO_FILE);
//destroy GUI windows
destroyAllWindows();
return EXIT_SUCCESS;
}
void processVideo(char* videoFilename) {
//create the capture object VideoCapture capture(videoFilename); if (!capture.isOpened()) { //error in opening the video input cerr << "Unable to open video file: " << videoFilename << endl; exit(EXIT_FAILURE); }
///read input data. ESC or 'q' for quitting while ((char)keyboard != 'q' && (char)keyboard != 27) { //read the current frame if (!capture.read(frame)) { cerr << "Unable to read next frame." << endl; cerr << "Exiting..." << endl; exit(EXIT_FAILURE); }
//update the background model pMOG2->apply(frame, fgMaskMOG2);
stringstream ss; rectangle(frame, cv::Point(10, 2), cv::Point(100, 20), cv::Scalar(255, 255, 255), -1); ss << capture.get(CAP_PROP_POS_FRAMES); string frameNumberString = ss.str(); putText(frame, frameNumberString.c_str(), cv::Point(15, 15), FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0, 0));
imshow("Frame", frame);
imshow("FG Mask MOG 2", fgMaskMOG2);
pMOG2->apply(frame, fgMaskMOG2);
imshow("Frame", frame);
imshow("FG Mask MOG 2", fgMaskMOG2);
//get the frame number and write it on the current frame rectangle(frame, Point(10, 2), Point(100, 20), Scalar(255, 255, 255), -1); frameNumberCurrent << capture.get(CAP_PROP_POS_FRAMES); frameInfo << frameNumberCurrent.str() << "/" << frameNumberTotal.str(); putText(frame, frameInfo.str(), Point(15, 15), FONT_HERSHEY_SIMPLEX, 0.5, Scalar(0, 0, 0));
//get the input from the keyboard keyboard = waitKey(30);
/// variable declaration and initialization
// morphing related variables (initialized during each loop, since parameters are changable)
Mat kernelClose;
Mat kernelErode;
Mat kernelDilate;
// frame related variables
stringstream frameNumberTotal;
stringstream frameNumberCurrent;
stringstream frameInfo;
int frameWidth = 0,
frameHeight = 0;
// contour related variables
vector<vector<Point>> contours;
vector<Vec4i> hierarchy;
// displaying related variables
int r = 1; // counter for row
//// video processing
frameNumberTotal << capture.get(CAP_PROP_FRAME_COUNT);
frameWidth = (int)capture.get(CV_CAP_PROP_FRAME_WIDTH);
frameHeight = (int)capture.get(CV_CAP_PROP_FRAME_HEIGHT);
if (frameEnd == 0)
frameEnd = (int)capture.get(CAP_PROP_FRAME_COUNT);
// set video to defined start frame
capture.set(CAP_PROP_POS_FRAMES, frameStart);
// show control panel (enables modification of some variables in realtime)
showControlPanel();
while ((char)keyboard != 27) {
// read the current frame
if (!capture.read(frame)) {
cerr << "Unable to read next frame." << endl;
cerr << "Exiting..." << endl;
exit(EXIT_FAILURE);
}
// convert original frame to grayscale if specified by constant
if (SOURCE2GRAY)
frameGray.copyTo(frame);
// re-initialize objects with changable parameters
if (kernelSizeClose > 0)
kernelClose = getStructuringElement(KERNEL_TYPE, Size(kernelSizeClose, kernelSizeClose));
if (kernelSizeErode > 0)
kernelErode = getStructuringElement(KERNEL_TYPE, Size(kernelSizeErode, kernelSizeErode));
if (kernelSizeDilate > 0)
kernelDilate = getStructuringElement(KERNEL_TYPE, Size(kernelSizeDilate, kernelSizeDilate));
}
// apply threshold to MOG2 frame to get rid of shadows etc
threshold(fgMaskMOG2, frameThreshold, thresholdMask, 255, THRESH_BINARY);
// apply morphology operations to thresholded frame
if (kernelSizeClose > 0)
morphologyEx(frameMorphed, frameMorphed, MORPH_CLOSE, kernelClose);
if (kernelSizeErode > 0)
morphologyEx(frameMorphed, frameMorphed, MORPH_ERODE, kernelErode);
if (kernelSizeDilate > 0)
morphologyEx(frameMorphed, frameMorphed, MORPH_DILATE, kernelDilate);
frameThreshold.copyTo(frameMorphed);
// save previous masked frame as grayscale frame for later tracking
if (!frameMasked.empty())
cvtColor(frameMasked, frameGrayPrev, CV_RGB2GRAY);
// apply morphed mask, but save before the morphed frame before
frameMorphed.copyTo(frameContours);
frameMorphed.copyTo(frameMasked);
frame.copyTo(frameMasked, frameMasked);
// save new masked frame as grayscale frame for later tracking, also save mask to KltTracking frame
cvtColor(frameMasked, frameGray, CV_RGB2GRAY);
frameMasked.copyTo(frameKltTracking);
// draw and fill contours
findContours(frameContours, contours, hierarchy, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
setIdentity(frameContours, Scalar(0, 0, 0));
drawContours(frameContours, contours, -1, Scalar(255, 255, 255), -1);
// draws rectangle around identified contours
drawBoundingBoxes(contours, frameMasked);
// prepare for and apply tracking algorithm (use grayscale copies of masked frame)
if (!frameGrayPrev.empty() && TRACKING)
applyKltTracking(frameGray, frameGrayPrev, frameKltTracking, contours);
/// show the different frames (from last to first row, to overlay windows bar)
resizeFrames(); // resize all frames
if (SHOW_ALL_OPERATION_RESULTS) {
// row 3
r = 3;
showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
// row 2
r = 2;
showFrame("Morphed", frameMorphed, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
showFrame("Contours", frameContours, frameWidth, frameHeight, 2 + (r - 1)*ELEMENTS_PER_ROW);
showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 3 + (r - 1)*ELEMENTS_PER_ROW);
// row 1
r = 1;
showFrame("Original", frame, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
showFrame("Foreground (MOG2)", fgMaskMOG2, frameWidth, frameHeight, 2 + (r - 1)*ELEMENTS_PER_ROW);
showFrame("Threshold", frameThreshold, frameWidth, frameHeight, 3 + (r - 1)*ELEMENTS_PER_ROW);
}
else {
// row 1
r = 1;
showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 2 + (r - 1)*ELEMENTS_PER_ROW);
}
// shows image for the specified amount of milliseconds
keyboard = waitKey(30);
switch (keyboard) {
case 'r':
resetFeatures = true; // reset on pressed 'r'
break;
}
// loop through the video if end is reached and loop is set
if (LOOP_VIDEO && (frameNumberCurrent.str().compare(std::to_string(frameEnd)) == 0))
capture.set(CAP_PROP_POS_FRAMES, frameStart);
// ensure that new feature points are initialized every INIT_INTERVAL frames for the tracker
if (std::stoi(frameNumberCurrent.str()) % INIT_INTERVAL == 0 && trackedObjects.size() > 0)
resetFeatures = true;
// cleaning operation: clean string streams
frameNumberCurrent.str("");
frameInfo.str("");
}
// cleaning operation: release capture object
capture.release();
//imshow("FG Mask MOG 2", fgMaskMOG2);
//get the input from the keyboard
//keyboard = waitKey(30);
//}
//delete capture object
////capture.release();
}
char* videoChooser() {
// variable declaration
char* videoFile = "dog.avi"; // saves name of selected video file
// chooses video file depending on chosen video number
switch (VIDEO_NUMBER) {
case 1:
videoFile = "visiontraffic.avi";
frameStart = 90;
frameEnd = 440;
break;
case 2:
videoFile = "viptraffic.avi";
break;
case 3:
videoFile = "highwayII_raw.avi";
break;
case 4:
videoFile = "dispersed.ogv";
break;
case 5:
videoFile = "highspeed.ogv";
break;
case 6:
videoFile = "hightraffic-dark.ogv";
break;
case 7:
videoFile = "hightraffic-occlude.ogv";
break;
case 8:
videoFile = "threecars.ogv";
break;
case 9:
videoFile = "turntraffic-dark.ogv";
break;
case 10:
videoFile = "visor_1261565143617_cesta1.wmv";
break;
default:
break;
}
return videoFile;
}
void showFrame(char* windowName, Mat frame, int width, int height, int position) { // variable declaration and initialization int x = 0, // x-coordinates for position y = 0, // y-coordinates for position yOffset = 30; // offset for y-coordinates, since windows bar does not count into frame height // strategy for y-offset: create windows from last row to first to overlay the bar
// create y-Offset if defince to make titles of windows readable
if (!WINDOW_NAMES_READABLE) {
yOffset += -32;
}
else if (position > ELEMENTS_PER_ROW) {
yOffset += 32;
}
// create window and display defined frame
cv::namedWindow(windowName);
cv::imshow(windowName, frame);
// resize stats for position calculation, if defined
if (scale != 10) {
// filter for the case, that it's zero, since range is not adjustable in opencv gui
if (scale == 0)
scale = 1;
// calculate scalefactor from value of scale variable
float scaleFactor = (float)scale / 10;
// calculate new width and height
width = (int)(width * scaleFactor);
height = (int)(height * scaleFactor);
}
//// get positioning
// get x coordinates with width
if (((position - 1) % ELEMENTS_PER_ROW) != 0)
x = ((position - 1) % ELEMENTS_PER_ROW) * width;
// get y coordinates with height
if (position > ELEMENTS_PER_ROW)
y = (position - (position - 1) % ELEMENTS_PER_ROW) / ELEMENTS_PER_ROW * height;
// place window at calculated position
moveWindow(windowName, x, y + yOffset);
}
void resizeFrames() { //resize all frames, if defined if (scale != 10 && scale > 0) { // filter for the case, that it's zero, since range is not adjustable in opencv gui if (scale == 0) scale = 1;
// calculate scalefactor from value of scale variable
float scaleFactor = (float)scale / 10;
// resize all frames
resize(frame, frame, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
resize(fgMaskMOG2, fgMaskMOG2, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
resize(frameThreshold, frameThreshold, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
resize(frameMorphed, frameMorphed, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
resize(frameMasked, frameMasked, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
resize(frameContours, frameContours, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
resize(frameKltTracking, frameKltTracking, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR);
}
}
void drawBoundingBoxes(vector<vector
// create bounding rect of object and draw on defined frame
for (vector<vector<Point>>::iterator itc = contours.begin(); itc != contours.end(); itc++) {
// variable declaration and initialization
Rect boundingBox = boundingRect(Mat(*itc));
Mat overlay;
// only handle rectangles with a minimum area
if (boundingBox.area() >= CONTOUR_MIN_AREA) {
if (contourArea(*itc) >= CONTOUR_MIN_AREA) {
// get corresponding object, if there's one, to define the drawing color
size_t* indexCorrespondingObject = getCorrespondingObject(*itc);
if (indexCorrespondingObject != NULL)
drawingColor = trackedObjects[*indexCorrespondingObject].getColor();
// draw contours regardless of area
drawContours(frame, contours, -1, drawingColor, 1);
// handle bounding boxes (but only if they're bigger than defined minimum area)
if (boundingBox.area() >= BBOX_MIN_AREA) {
// copy frame to
frame.copyTo(overlay);
// draw rectangle
rectangle(overlay, boundingBox, drawingColor, 3);
// draw test label if constant is set
if (SHOW_LABELS)
rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y),
Point(boundingBox.x + boundingBox.width + 1,
boundingBox.y - LABEL_HEIGHT), drawingColor, -1);
// combine transparent overlay of bounding rect with original frame
cv::addWeighted(overlay, ALPHA, frame, 1 - ALPHA, 0, frame);
// afterwards, put text with object name into the label, if constant is set
if (SHOW_LABELS)
putText(frame, TEST_NAME, Point(boundingBox.x + 10, boundingBox.y - 2),
TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(0, 0, 0),
TEXT_THICKNESS, TEXT_LINE_TYPE, false);
// count handled contours
amountHandledContours++;
}
}
}
}
// re-initialize feature tracker if new objects appeared in this frame
if (amountHandledContours > amountHandledContoursPrev) {
resetFeatures = true;
cout << amountHandledContours << "/" << amountHandledContoursPrev << endl;
}
amountHandledContoursPrev = amountHandledContours;
}
void showControlPanel() { // constant declaration const int WIDTH = 400, HEIGHT = 240;
const char* nameWindow = "Control Panel";
const char* nameTrackbarWindowScale = "Scale";
const char* nameTrackbarThreshold = "Threshold";
const char* nameTrackbarKernelSizeClose = "SizeClose";
const char* nameTrackbarKernelSizeErode = "SizeErode";
const char* nameTrackbarKernelSizeDilate = "SizeDilate";
// create window
namedWindow(nameWindow, CV_WINDOW_AUTOSIZE | CV_GUI_NORMAL);
resizeWindow(nameWindow, WIDTH, HEIGHT);
// window relating parameters
createTrackbar(nameTrackbarWindowScale, nameWindow, &scale, 20, NULL);
createTrackbar(nameTrackbarThreshold, nameWindow, &thresholdMask, THRESHOLD_MASK_MAX, NULL);
createTrackbar(nameTrackbarKernelSizeClose, nameWindow, &kernelSizeClose, KERNEL_SIZE_MAX, NULL);
createTrackbar(nameTrackbarKernelSizeErode, nameWindow, &kernelSizeErode, KERNEL_SIZE_MAX, NULL);
createTrackbar(nameTrackbarKernelSizeDilate, nameWindow, &kernelSizeDilate, KERNEL_SIZE_MAX, NULL);
}
/**
@function for klt tracking
*/
void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector
/// variable declaration
vector
/// calculate features and then the optical flow if (resetFeatures) { // automatic initialization goodFeaturesToTrack(frameGray, newPoints, MAX_COUNT_TOTAL, 0.01, 10, Mat(), 3, 0, 0.04); cornerSubPix(frameGray, newPoints, subPixWinSize, Size(-1, -1), termcrit);
//points[0].clear(); // could solve performance issue, but creates strange behaviour:
// trackedObjects also reset every reset
//points[1].clear();
// initialize with all available points from tracked objects
for (std::vector<TrackedObject>::size_type l = 0; l != trackedObjects.size(); l++)
for (std::vector<TrackedObject>::size_type j = 0; j != trackedObjects[l].getPoints().size(); j++) {
points[0].push_back(trackedObjects[l].getPoints()[j]);
}
}
if (!points[0].empty() || trackedObjects.size() > 0) {
// variable declaration
vector
// check if previous frame is existing, otherwise copy the current one
if (frameGrayPrev.empty())
frameGray.copyTo(frameGrayPrev);
// calculate the optical flow with KLT
calcOpticalFlowPyrLK(frameGrayPrev, frameGray, points[0], points[1], status, err, winSize,
3, termcrit, 0, 0.001);
/// iterate through all found points
for (i = k = 0; i < points[1].size(); i++) {
// don't handle point, if there's no corresponding point from the last frame
if (!status[i])
continue;
// otherwise save and draw point (shrink's point vector, resizes after the for loop)
points[1][k++] = points[1][i];
/// get corresponding contour and object, otherwise create new tracked object
if (STATEFUL_TRACKING)
trackObjects(contours, i);
else
circle(frameKltTracking, points[1][i], 6, DEFAULT_DRAWING_COLOR, 2, 8);
}
points[1].resize(k);
}
// handle the processed points handleProcessedPoints();
// save that initialization was done and not needed anymore, also add new points if (resetFeatures) { resetFeatures = false;
for (std::vector<Point2f>::size_type l = 0; l != newPoints.size(); l++)
points[1].push_back(newPoints[l]);
}
// save the current values as the previous ones if (!resetFeatures) std::swap(points[1], points[0]);
cv::swap(frameGrayPrev, frameGrayPrev); }
void trackObjects(vector<vector
/// get contour of point
for (m = 0; m != contours.size(); m++) {
// checks if point is inside or on the edge of the contour
containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false);
// exit loop if corresponding contour is found
if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) {
foundContour = true;
break;
}
}
if (trackedObjects.size() > 0) {
// check for each point in the tracked object if it already exists
for (std::vector<TrackedObject>::size_type l = 0; l != trackedObjects.size(); l++) {
size_t* containedInObject = NULL;
if (trackedObjects[l].containsOldPoint(points[0][currentPoint]) != NULL) {
// add the point (it will increase weight automatically instead)
trackedObjects[l].addPoint(points[1][currentPoint], points[0][currentPoint]);
foundObject = true;
// update contour of object, if contour was not modified this frame yet
bool foundContour = false;
if (!trackedObjects[l].getContourStatus()) {
std::vector<Point>::size_type m;
for (m = 0; m != contours.size(); m++) {
// checks if point is inside or on the edge of the contour
containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false);
// exit loop if corresponding contour is found
if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) {
foundContour = true;
break;
}
}
// update corresponding contour, if one was found
if (foundContour)
trackedObjects[l].setContour(contours[m]);
}
}
}
}
// create new object with this point
if (!foundObject && foundContour) {
size_t* containedInObject;
containedInObject = getCorrespondingObject(contours[m]);
if (containedInObject != NULL) {
// contour belongs to an existing object, so add the point
size_t containedInObjectVal = *containedInObject;
// add points
if (trackedObjects[containedInObjectVal].getPoints().size() < MAX_COUNT_PER_CAR)
trackedObjects[containedInObjectVal].addPoint(points[1][currentPoint], points[0][currentPoint]);
foundObjectOfContour = true;
}
// create new tracked object, since there's no existing one with this contour
if (!foundObjectOfContour) {
char name[11] = "Object_";
char objectNumber[4];
sprintf(objectNumber, "%d", (int)(objectCounter + 1));
strcat(name, objectNumber);
TrackedObject* trObj = new TrackedObject(name);
trObj->addPoint(points[1][currentPoint], points[0][currentPoint]);
trObj->setContour(contours[m]);
trackedObjects.push_back(*trObj);
objectCounter++;
}
}
}
size_t getCorrespondingObject(vector
// find out if contour is assigned to existing object
if (trackedObjects.size() > 0 && contour.size() > 0) {
for (std::vector<TrackedObject>::size_type i = 0; i != trackedObjects.size(); i++) {
if (trackedObjects[i].getContour().size() == contour.size()) {
double comparisonValue;
comparisonValue = matchShapes(trackedObjects[i].getContour(), contour, CV_CONTOURS_MATCH_I1, NULL);
if (comparisonValue == 0) {
position = &i;
break;
}
}
}
}
return position;
}
void handleProcessedPoints() { // get original frame, if constant defines, that results should be shown in original if (TRACKING_RESULTS_IN_ORIGINAL) frame.copyTo(frameKltTracking);
if (trackedObjects.size() > 0) {
for (std::vector<TrackedObject>::size_type i = 0; i != trackedObjects.size(); i++) {
// process all points according to the status
trackedObjects[i].processStatus();
if (trackedObjects[i].getPoints().size() <= MIN_POINTS_TRACKED_OBJECT) {
trackedObjects.erase(trackedObjects.begin() + i);
i--;
continue;
}
/// draw bounding box
Rect boundingBox = boundingRect(trackedObjects[i].getContour());
Mat overlay;
frameKltTracking.copyTo(overlay);
/// draw rectangle and label
rectangle(overlay, boundingBox, trackedObjects[i].getColor(), 3);
rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y),
Point(boundingBox.x + boundingBox.width + 1,
boundingBox.y - LABEL_HEIGHT), trackedObjects[i].getColor(), -1);
// combine transparent overlay of bounding rect with original frame
cv::addWeighted(overlay, ALPHA, frameKltTracking, 1 - ALPHA, 0, frameKltTracking);
// afterwards, put text with object name into the label, if constant is set
putText(frameKltTracking, trackedObjects[i].getName(), Point(boundingBox.x + 10, boundingBox.y - 2),
TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(200, 200, 200),
TEXT_THICKNESS, TEXT_LINE_TYPE, false);
/// draw all feature points in the color of the corresponding object
for (std::vector<Point2f>::size_type j = 0; j != trackedObjects[i].getPoints().size(); j++) {
// handle weight of point
int w = trackedObjects[i].getWeight()[j];
int size = (int)trackedObjects[i].getPoints().size();
if (w != -1) { // point not found
// draw filled circle for new points
if (w == 1)
w = -1;
else if (w >= HEATMAP_RANGE)
w = HEATMAP_RANGE;
if (w != -1)
circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, getHeatMapColor(1 - (float)w / (float)HEATMAP_RANGE), -1, 8);
circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, trackedObjects[i].getColor(), 2, 8);
}
}
}
}
}
Scalar getHeatMapColor(float value) { float red, green, blue; const int NUM_COLORS = 4; static float color[NUM_COLORS][3] = { { 0, 0, 1 }, { 0, 1, 0 }, { 1, 1, 0 }, { 1, 0, 0 } }; // A static array of 4 colors: (blue, green, yellow, red) using {r,g,b} for each.
int idx1; // |-- Our desired color will be between these two indexes in "color".
int idx2; // |
float fractBetween = 0; // Fraction between "idx1" and "idx2" where our value is.
if (value <= 0) { idx1 = idx2 = 0; } // accounts for an input <=0
else if (value >= 1) { idx1 = idx2 = NUM_COLORS - 1; } // accounts for an input >=0
else
{
value = value * (NUM_COLORS - 1);
idx1 = (int)floor(value); // Our desired color will be after this index.
idx2 = idx1 + 1; // ... and before this index (inclusive).
fractBetween = value - float(idx1); // Distance between the two indexes (0-1).
}
red = ((color[idx2][0] - color[idx1][0])*fractBetween + color[idx1][0]) * 255;
green = ((color[idx2][1] - color[idx1][1])*fractBetween + color[idx1][1]) * 255;
blue = ((color[idx2][2] - color[idx1][2])*fractBetween + color[idx1][2]) * 255;
return Scalar(red, green, blue);
}
Hi, still not clear what do you want, this code is for MOG2, it uses MOG2, and if it works, than no bugs here.
2017-08-23 7:10 GMT+03:00 dulithaaaa notifications@github.com:
Thank you very much sir,sir i have code can you find what is bug in the code please,it works only to MOG2
On Wednesday, August 23, 2017 9:16 AM, Sergey Nuzhny < notifications@github.com> wrote:
Sir,I have a code of tracking i attached that file.it worked only to MOG2,could you please look at that code and tell me what modification needs to be required. Hi! I don't know what do you want. How can I advise you? This project - Multitarget-tracker contains full stack of algorithms for motion detection. I can tell you how it works.
- You need CMake gui application, start it.
- Select folder Mulitarget-tracker (with CMakeLists.txt file).
- Select subfolder folder build.
- Push a button Configure, select your MSVC compler. Wait.
- If OpenCV wasn't founded automatic then select OpenCV_DIR manually. Push a button Configure again.
- Push a button Generate.
- Now the folder build contains a solution file (.sln). Lets open it in Visual studio! The end. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.
include "opencv2/imgcodecs.hpp"
include "opencv2/imgproc.hpp"
include "opencv2/videoio.hpp"
include <opencv2/highgui.hpp>
include <opencv2/video.hpp>
include
include
include
include
include "TrackedObject.h"
using namespace cv; using namespace std;
// Global constant
const bool SHOW_ALL_OPERATION_RESULTS = true; // defines if different operation results should be shown const bool WINDOW_NAMES_READABLE = false; // if set, creates y-offset to make window names readable const int ELEMENTS_PER_ROW = 3; // defines how much windows are shown within a row const bool TRACKING_RESULTS_IN_ORIGINAL = false; // shows tracking results in original frame
// video related constants const int VIDEO_NUMBER = 1; // 1:10, choose different number for different video const bool LOOP_VIDEO = true; // defines if video should be looped infinitely
const char* VIDEO_DIRECTORY_PATH = ".\detect.avi\"; // define path to video directory
const bool SOURCE2GRAY = false; // defines, if source frame should be converted to gray
// foreground segmentation postprocessing related constants const int HISTORY = 1000; // MOG2 parameter: amount of considered frames const double VAR_THRESHOLD = 20; // MOG2 parameter: threshold for degree of minimum variance const bool DETECT_SHADOWS = false; // MOG2 parameter: defines if shadows should be detected
int thresholdMask = 100; // defines start value of threshold for // binarization of foreground frame const int THRESHOLD_MASK_MAX = 255; // defines maximum value for threshold
// morphing related constants const int KERNEL_TYPE = MORPH_ELLIPSE; // choose: MORPH_RECT, MORPH_CROSS, MORPH_ELLIPSE int kernelSizeClose = 5; // defines start value for kernel size of close op. int kernelSizeErode = 5; // defines start value for kernel size of erode op. int kernelSizeDilate = 5; // defines start value for kernel size of dilate op. const int KERNEL_SIZE_MAX = 20; // defines maximum value for threshold
// drawing related constants const Scalar DEFAULT_DRAWING_COLOR = Scalar(0, 255, 0); // defines default color of bounding boxes and labels const int CONTOUR_MIN_AREA = 500; // defines minimum area of contours const int BBOX_MIN_AREA = 3000; // defines minimum area of bounding boxes const bool SHOW_LABELS = false; // shows labels of object adjusted to the bounding box const int HEATMAP_RANGE = 25; // defines size of different colors in heat map // and threshold (color of highest weight)
// bounding box and label related constants const float ALPHA = 0.5; // defines opacity of rectangle and label const int LABEL_HEIGHT = 20; // defines height of label const char* TEST_NAME = "Car"; // defines name for test labels
const int TEXT_FONT_FACE = 2; // defines font of label text const int TEXT_THICKNESS = 2; // defines thickness of label text const int TEXT_LINE_TYPE = 0; // defines line type of label text const double TEXT_FONT_SCALE = 0.7; // defines scale of label text
// tracking related constants const bool TRACKING = true; // defines if tracking is active const bool STATEFUL_TRACKING = true; // defines if stateful tracking with TrackedObjects is active const int MAX_COUNT_TOTAL = 150; // defines max no. of tracked points in total const int MAX_COUNT_PER_CAR = 25; // defines max no. of tracked points per car const int MIN_POINTS_TRACKED_OBJECT = 3; // defines min no. of points for tracked objects const int INIT_INTERVAL = 15; // defines, after how many frames the tracker should initialize new points const TermCriteria termcrit(TermCriteria::COUNT | TermCriteria::EPS, 20, 0.03); // defines criteria for klt tracker const Size subPixWinSize(10, 10); const Size winSize(31, 31);
/// Global variables
// window related variables int scale = 10; // scales windows (and mats) after processing before displaying // (10 is equal to standard size, 20 to doubled size)
// video related variables int frameStart = 1; // defines frame to start (set in video chooser) int frameEnd = 0; // defines frame to end (set in video chooser)
// frame related variables
Mat frame; //current frame Mat fgMaskMOG2; //fg mask fg mask generated by MOG2 method Mat frameGray; // current frame in grayscale Mat frameGrayPrev; // previous frame in grayscale Mat frameThreshold; // frame with applied threshold (to elimante shadows etc.) Mat frameMorphed; // morphed frame (to elimante noise) Mat frameMasked; // morphed frame applied as a mask to the original frame Mat frameContours; // drawn contours Mat frameKltTracking; // klt tracking lanes
Ptr
pMOG2; //MOG2 Background subtractor // tracking related variables bool resetFeatures = false; Point2f point; vector
points[2]; int amountHandledContoursPrev = 0; vector
trackedObjects; int objectCounter; int keyboard; //input from keyboard
/// Function Headers
char* videoChooser();
void processVideo(char* videoFilename);
void showFrame(char* windowName, Mat frame, int width,int height, int position);
void resizeFrames();
void drawBoundingBoxes(vector<vector
> contours, Mat frame); void showControlPanel();
void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector
> contours); void trackObjects(vector<vector
> contours, size_t currentPoint); size_t* getCorrespondingObject(vector
contour); void handleProcessedPoints();
Scalar getHeatMapColor(float value);
void processVideo(char videoFilename); void processImages(char firstFrameFilename);
char* VIDEO_FILE = "visiontraffic.avi";
int main(int argc, char* argv[]) {
char* videoPath = new char[100];
/// initialize variables // define path of video file strcpy(videoPath, VIDEO_DIRECTORY_PATH); strcat(videoPath, videoChooser());
//create GUI windows namedWindow("Original"); namedWindow("Foreground (MOG2)"); namedWindow("Threshold"); namedWindow("Morphed"); namedWindow("Contours"); namedWindow("Masked Original Frame"); namedWindow("KLT Tracking");
//create Background Subtractor objects pMOG2 = createBackgroundSubtractorMOG2(HISTORY, VAR_THRESHOLD, DETECT_SHADOWS); //MOG2 approach
//input data coming from a video processVideo(VIDEO_FILE); //destroy GUI windows destroyAllWindows(); return EXIT_SUCCESS; }
void processVideo(char* videoFilename) {
//create the capture object VideoCapture capture(videoFilename); if (!capture.isOpened()) { //error in opening the video input cerr << "Unable to open video file: " << videoFilename << endl; exit(EXIT_FAILURE); }
///read input data. ESC or 'q' for quitting while ((char)keyboard != 'q' && (char)keyboard != 27) { //read the current frame if (!capture.read(frame)) { cerr << "Unable to read next frame." << endl; cerr << "Exiting..." << endl; exit(EXIT_FAILURE); }
//update the background model pMOG2->apply(frame, fgMaskMOG2);
stringstream ss; rectangle(frame, cv::Point(10, 2), cv::Point(100, 20), cv::Scalar(255, 255, 255), -1); ss << capture.get(CAP_PROP_POS_FRAMES); string frameNumberString = ss.str(); putText(frame, frameNumberString.c_str(), cv::Point(15, 15), FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0, 0));
imshow("Frame", frame); imshow("FG Mask MOG 2", fgMaskMOG2);
pMOG2->apply(frame, fgMaskMOG2);
imshow("Frame", frame); imshow("FG Mask MOG 2", fgMaskMOG2);
//get the frame number and write it on the current frame rectangle(frame, Point(10, 2), Point(100, 20), Scalar(255, 255, 255), -1); frameNumberCurrent << capture.get(CAP_PROP_POS_FRAMES); frameInfo << frameNumberCurrent.str() << "/" << frameNumberTotal.str(); putText(frame, frameInfo.str(), Point(15, 15), FONT_HERSHEY_SIMPLEX, 0.5, Scalar(0, 0, 0));
//get the input from the keyboard keyboard = waitKey(30);
/// variable declaration and initialization // morphing related variables (initialized during each loop, since parameters are changable) Mat kernelClose; Mat kernelErode; Mat kernelDilate;
// frame related variables stringstream frameNumberTotal; stringstream frameNumberCurrent; stringstream frameInfo; int frameWidth = 0, frameHeight = 0;
// contour related variables vector<vector
> contours; vector hierarchy; // displaying related variables int r = 1; // counter for row
//// video processing
frameNumberTotal << capture.get(CAP_PROP_FRAME_COUNT); frameWidth = (int)capture.get(CV_CAP_PROP_FRAME_WIDTH); frameHeight = (int)capture.get(CV_CAP_PROP_FRAME_HEIGHT);
if (frameEnd == 0) frameEnd = (int)capture.get(CAP_PROP_FRAME_COUNT);
// set video to defined start frame capture.set(CAP_PROP_POS_FRAMES, frameStart);
// show control panel (enables modification of some variables in realtime) showControlPanel();
while ((char)keyboard != 27) { // read the current frame if (!capture.read(frame)) { cerr << "Unable to read next frame." << endl; cerr << "Exiting..." << endl; exit(EXIT_FAILURE); }
// convert original frame to grayscale if specified by constant if (SOURCE2GRAY) frameGray.copyTo(frame);
// re-initialize objects with changable parameters if (kernelSizeClose > 0) kernelClose = getStructuringElement(KERNEL_TYPE, Size(kernelSizeClose, kernelSizeClose)); if (kernelSizeErode > 0) kernelErode = getStructuringElement(KERNEL_TYPE, Size(kernelSizeErode, kernelSizeErode)); if (kernelSizeDilate > 0) kernelDilate = getStructuringElement(KERNEL_TYPE, Size(kernelSizeDilate, kernelSizeDilate));
} // apply threshold to MOG2 frame to get rid of shadows etc threshold(fgMaskMOG2, frameThreshold, thresholdMask, 255, THRESH_BINARY);
// apply morphology operations to thresholded frame
if (kernelSizeClose > 0) morphologyEx(frameMorphed, frameMorphed, MORPH_CLOSE, kernelClose); if (kernelSizeErode > 0) morphologyEx(frameMorphed, frameMorphed, MORPH_ERODE, kernelErode); if (kernelSizeDilate > 0) morphologyEx(frameMorphed, frameMorphed, MORPH_DILATE, kernelDilate);
frameThreshold.copyTo(frameMorphed);
// save previous masked frame as grayscale frame for later tracking if (!frameMasked.empty()) cvtColor(frameMasked, frameGrayPrev, CV_RGB2GRAY);
// apply morphed mask, but save before the morphed frame before frameMorphed.copyTo(frameContours); frameMorphed.copyTo(frameMasked); frame.copyTo(frameMasked, frameMasked);
// save new masked frame as grayscale frame for later tracking, also save mask to KltTracking frame cvtColor(frameMasked, frameGray, CV_RGB2GRAY); frameMasked.copyTo(frameKltTracking);
// draw and fill contours findContours(frameContours, contours, hierarchy, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE); setIdentity(frameContours, Scalar(0, 0, 0)); drawContours(frameContours, contours, -1, Scalar(255, 255, 255), -1);
// draws rectangle around identified contours drawBoundingBoxes(contours, frameMasked);
// prepare for and apply tracking algorithm (use grayscale copies of masked frame) if (!frameGrayPrev.empty() && TRACKING) applyKltTracking(frameGray, frameGrayPrev, frameKltTracking, contours);
/// show the different frames (from last to first row, to overlay windows bar) resizeFrames(); // resize all frames
if (SHOW_ALL_OPERATION_RESULTS) { // row 3 r = 3; showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
// row 2 r = 2; showFrame("Morphed", frameMorphed, frameWidth, frameHeight, 1 + (r - 1)ELEMENTS_PER_ROW); showFrame("Contours", frameContours, frameWidth, frameHeight, 2 + (r - 1)ELEMENTS_PER_ROW); showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 3
- (r - 1)*ELEMENTS_PER_ROW);
// row 1 r = 1; showFrame("Original", frame, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW); showFrame("Foreground (MOG2)", fgMaskMOG2, frameWidth, frameHeight, 2 + (r
- 1)ELEMENTS_PER_ROW); showFrame("Threshold", frameThreshold, frameWidth, frameHeight, 3 + (r - 1)ELEMENTS_PER_ROW);
} else { // row 1 r = 1; showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 1
- (r - 1)ELEMENTS_PER_ROW); showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 2 + (r - 1)ELEMENTS_PER_ROW); }
// shows image for the specified amount of milliseconds keyboard = waitKey(30);
switch (keyboard) { case 'r': resetFeatures = true; // reset on pressed 'r' break; }
// loop through the video if end is reached and loop is set if (LOOP_VIDEO && (frameNumberCurrent.str().compare(std::to_string(frameEnd)) == 0)) capture.set(CAP_PROP_POS_FRAMES, frameStart);
// ensure that new feature points are initialized every INIT_INTERVAL frames for the tracker if (std::stoi(frameNumberCurrent.str()) % INIT_INTERVAL == 0 && trackedObjects.size() > 0) resetFeatures = true;
// cleaning operation: clean string streams frameNumberCurrent.str(""); frameInfo.str(""); }
// cleaning operation: release capture object capture.release();
//imshow("FG Mask MOG 2", fgMaskMOG2);
//get the input from the keyboard //keyboard = waitKey(30); //} //delete capture object ////capture.release(); }
char* videoChooser() {
// variable declaration char* videoFile = "dog.avi"; // saves name of selected video file
// chooses video file depending on chosen video number switch (VIDEO_NUMBER) { case 1: videoFile = "visiontraffic.avi"; frameStart = 90; frameEnd = 440; break;
case 2: videoFile = "viptraffic.avi"; break;
case 3: videoFile = "highwayII_raw.avi"; break;
case 4: videoFile = "dispersed.ogv"; break;
case 5: videoFile = "highspeed.ogv"; break;
case 6: videoFile = "hightraffic-dark.ogv"; break;
case 7: videoFile = "hightraffic-occlude.ogv"; break;
case 8: videoFile = "threecars.ogv"; break;
case 9: videoFile = "turntraffic-dark.ogv"; break;
case 10: videoFile = "visor_1261565143617_cesta1.wmv"; break;
default: break; }
return videoFile; }
void showFrame(char* windowName, Mat frame, int width, int height, int position) { // variable declaration and initialization int x = 0, // x-coordinates for position y = 0, // y-coordinates for position yOffset = 30; // offset for y-coordinates, since windows bar does not count into frame height // strategy for y-offset: create windows from last row to first to overlay the bar
// create y-Offset if defince to make titles of windows readable if (!WINDOW_NAMES_READABLE) { yOffset += -32; } else if (position > ELEMENTS_PER_ROW) { yOffset += 32; }
// create window and display defined frame
cv::namedWindow(windowName); cv::imshow(windowName, frame);
// resize stats for position calculation, if defined if (scale != 10) { // filter for the case, that it's zero, since range is not adjustable in opencv gui if (scale == 0) scale = 1;
// calculate scalefactor from value of scale variable float scaleFactor = (float)scale / 10;
// calculate new width and height width = (int)(width scaleFactor); height = (int)(height scaleFactor); }
//// get positioning
// get x coordinates with width if (((position - 1) % ELEMENTS_PER_ROW) != 0) x = ((position - 1) % ELEMENTS_PER_ROW) * width;
// get y coordinates with height if (position > ELEMENTS_PER_ROW) y = (position - (position - 1) % ELEMENTS_PER_ROW) / ELEMENTS_PER_ROW * height;
// place window at calculated position moveWindow(windowName, x, y + yOffset); }
void resizeFrames() { //resize all frames, if defined if (scale != 10 && scale > 0) { // filter for the case, that it's zero, since range is not adjustable in opencv gui if (scale == 0) scale = 1;
// calculate scalefactor from value of scale variable float scaleFactor = (float)scale / 10;
// resize all frames resize(frame, frame, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(fgMaskMOG2, fgMaskMOG2, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameThreshold, frameThreshold, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameMorphed, frameMorphed, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameMasked, frameMasked, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameContours, frameContours, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameKltTracking, frameKltTracking, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); } }
void drawBoundingBoxes(vector<vector
> contours, Mat frame) { // variable declaration int amountHandledContours = 0; Scalar drawingColor = DEFAULT_DRAWING_COLOR; // create bounding rect of object and draw on defined frame for (vector<vector
>::iterator itc = contours.begin(); itc != contours.end(); itc++) { // variable declaration and initialization Rect boundingBox = boundingRect(Mat(*itc)); Mat overlay; // only handle rectangles with a minimum area if (boundingBox.area() >= CONTOUR_MIN_AREA) { if (contourArea(*itc) >= CONTOUR_MIN_AREA) {
// get corresponding object, if there's one, to define the drawing color size_t indexCorrespondingObject = getCorrespondingObject(itc); if (indexCorrespondingObject != NULL) drawingColor = trackedObjects[*indexCorrespondingObject].getColor();
// draw contours regardless of area drawContours(frame, contours, -1, drawingColor, 1);
// handle bounding boxes (but only if they're bigger than defined minimum area) if (boundingBox.area() >= BBOX_MIN_AREA) { // copy frame to frame.copyTo(overlay);
// draw rectangle rectangle(overlay, boundingBox, drawingColor, 3);
// draw test label if constant is set if (SHOW_LABELS) rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y), Point(boundingBox.x + boundingBox.width + 1, boundingBox.y - LABEL_HEIGHT), drawingColor, -1);
// combine transparent overlay of bounding rect with original frame cv::addWeighted(overlay, ALPHA, frame, 1 - ALPHA, 0, frame);
// afterwards, put text with object name into the label, if constant is set if (SHOW_LABELS) putText(frame, TEST_NAME, Point(boundingBox.x + 10, boundingBox.y - 2), TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(0, 0, 0), TEXT_THICKNESS, TEXT_LINE_TYPE, false);
// count handled contours amountHandledContours++; } } } }
// re-initialize feature tracker if new objects appeared in this frame if (amountHandledContours > amountHandledContoursPrev) { resetFeatures = true; cout << amountHandledContours << "/" << amountHandledContoursPrev << endl; }
amountHandledContoursPrev = amountHandledContours; }
void showControlPanel() { // constant declaration const int WIDTH = 400, HEIGHT = 240;
const char nameWindow = "Control Panel"; const char nameTrackbarWindowScale = "Scale"; const char nameTrackbarThreshold = "Threshold"; const char nameTrackbarKernelSizeClose = "SizeClose"; const char nameTrackbarKernelSizeErode = "SizeErode"; const char nameTrackbarKernelSizeDilate = "SizeDilate";
// create window namedWindow(nameWindow, CV_WINDOW_AUTOSIZE | CV_GUI_NORMAL); resizeWindow(nameWindow, WIDTH, HEIGHT);
// window relating parameters createTrackbar(nameTrackbarWindowScale, nameWindow, &scale, 20, NULL); createTrackbar(nameTrackbarThreshold, nameWindow, &thresholdMask, THRESHOLD_MASK_MAX, NULL);
createTrackbar(nameTrackbarKernelSizeClose, nameWindow, &kernelSizeClose, KERNEL_SIZE_MAX, NULL); createTrackbar(nameTrackbarKernelSizeErode, nameWindow, &kernelSizeErode, KERNEL_SIZE_MAX, NULL); createTrackbar(nameTrackbarKernelSizeDilate, nameWindow, &kernelSizeDilate, KERNEL_SIZE_MAX, NULL); }
/**
- @function for klt tracking */ void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector
> contours) { /// variable declaration
vector
newPoints; /// calculate features and then the optical flow if (resetFeatures) { // automatic initialization goodFeaturesToTrack(frameGray, newPoints, MAX_COUNT_TOTAL, 0.01, 10, Mat(), 3, 0, 0.04); cornerSubPix(frameGray, newPoints, subPixWinSize, Size(-1, -1), termcrit);
//points[0].clear(); // could solve performance issue, but creates strange behaviour: // trackedObjects also reset every reset //points[1].clear();
// initialize with all available points from tracked objects for (std::vector
::size_type l = 0; l != trackedObjects.size(); l++) for (std::vector ::size_type j = 0; j != trackedObjects[l].getPoints().size(); j++) { points[0].push_back(trackedObjects[l].getPoints()[j]); } } if (!points[0].empty() || trackedObjects.size() > 0) { // variable declaration vector status; vector err; size_t i, k; // check if previous frame is existing, otherwise copy the current one if (frameGrayPrev.empty()) frameGray.copyTo(frameGrayPrev);
// calculate the optical flow with KLT calcOpticalFlowPyrLK(frameGrayPrev, frameGray, points[0], points[1], status, err, winSize, 3, termcrit, 0, 0.001);
/// iterate through all found points for (i = k = 0; i < points[1].size(); i++) {
// don't handle point, if there's no corresponding point from the last frame if (!status[i]) continue;
// otherwise save and draw point (shrink's point vector, resizes after the for loop) points[1][k++] = points[1][i];
/// get corresponding contour and object, otherwise create new tracked object if (STATEFUL_TRACKING) trackObjects(contours, i); else circle(frameKltTracking, points[1][i], 6, DEFAULT_DRAWING_COLOR, 2, 8); }
points[1].resize(k); }
// handle the processed points handleProcessedPoints();
// save that initialization was done and not needed anymore, also add new points if (resetFeatures) { resetFeatures = false;
for (std::vector
::size_type l = 0; l != newPoints.size(); l++) points[1].push_back(newPoints[l]); } // save the current values as the previous ones if (!resetFeatures) std::swap(points[1], points[0]);
cv::swap(frameGrayPrev, frameGrayPrev); }
void trackObjects(vector<vector
> contours, size_t currentPoint) { /// variable declaration int containedInContour; bool foundObject = false; bool foundContour = false; bool foundObjectOfContour = false; vector ::size_type m; // iterator for contours size_t* correspondingObject = NULL; // pointer for corresponding object /// get contour of point for (m = 0; m != contours.size(); m++) { // checks if point is inside or on the edge of the contour containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false);
// exit loop if corresponding contour is found if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) { foundContour = true; break; } }
if (trackedObjects.size() > 0) { // check for each point in the tracked object if it already exists for (std::vector
::size_type l = 0; l != trackedObjects.size(); l++) { size_t* containedInObject = NULL; if (trackedObjects[l].containsOldPoint(points[0][currentPoint]) != NULL) { // add the point (it will increase weight automatically instead) trackedObjects[l].addPoint(points[1][currentPoint], points[0][currentPoint]); foundObject = true;
// update contour of object, if contour was not modified this frame yet bool foundContour = false; if (!trackedObjects[l].getContourStatus()) { std::vector
::size_type m; for (m = 0; m != contours.size(); m++) { // checks if point is inside or on the edge of the contour containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false); // exit loop if corresponding contour is found if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) { foundContour = true; break; } }
// update corresponding contour, if one was found if (foundContour) trackedObjects[l].setContour(contours[m]); } }
} }
// create new object with this point if (!foundObject && foundContour) { size_t* containedInObject; containedInObject = getCorrespondingObject(contours[m]);
if (containedInObject != NULL) { // contour belongs to an existing object, so add the point size_t containedInObjectVal = *containedInObject;
// add points if (trackedObjects[containedInObjectVal].getPoints().size() < MAX_COUNT_PER_CAR) trackedObjects[containedInObjectVal].addPoint(points[1][currentPoint], points[0][currentPoint]);
foundObjectOfContour = true; }
// create new tracked object, since there's no existing one with this contour if (!foundObjectOfContour) { char name[11] = "Object_"; char objectNumber[4]; sprintf(objectNumber, "%d", (int)(objectCounter + 1)); strcat(name, objectNumber);
TrackedObject* trObj = new TrackedObject(name); trObj->addPoint(points[1][currentPoint], points[0][currentPoint]);
trObj->setContour(contours[m]);
trackedObjects.push_back(*trObj); objectCounter++; } } }
size_t getCorrespondingObject(vector
contour) { // variable declaration size_t position = NULL;// find out if contour is assigned to existing object if (trackedObjects.size() > 0 && contour.size() > 0) { for (std::vector
::size_type i = 0; i != trackedObjects.size(); i++) { if (trackedObjects[i].getContour().size() == contour.size()) { double comparisonValue; comparisonValue = matchShapes(trackedObjects[i].getContour(), contour, CV_CONTOURS_MATCH_I1, NULL); if (comparisonValue == 0) { position = &i; break; } } } } return position; }
void handleProcessedPoints() { // get original frame, if constant defines, that results should be shown in original if (TRACKING_RESULTS_IN_ORIGINAL) frame.copyTo(frameKltTracking);
if (trackedObjects.size() > 0) { for (std::vector
::size_type i = 0; i != trackedObjects.size(); i++) { // process all points according to the status trackedObjects[i].processStatus();
if (trackedObjects[i].getPoints().size() <= MIN_POINTS_TRACKED_OBJECT) { trackedObjects.erase(trackedObjects.begin() + i); i--; continue; }
/// draw bounding box Rect boundingBox = boundingRect(trackedObjects[i].getContour()); Mat overlay; frameKltTracking.copyTo(overlay);
/// draw rectangle and label rectangle(overlay, boundingBox, trackedObjects[i].getColor(), 3); rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y), Point(boundingBox.x + boundingBox.width + 1, boundingBox.y - LABEL_HEIGHT), trackedObjects[i].getColor(), -1);
// combine transparent overlay of bounding rect with original frame cv::addWeighted(overlay, ALPHA, frameKltTracking, 1 - ALPHA, 0, frameKltTracking);
// afterwards, put text with object name into the label, if constant is set putText(frameKltTracking, trackedObjects[i].getName(), Point(boundingBox.x
- 10, boundingBox.y - 2), TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(200, 200, 200), TEXT_THICKNESS, TEXT_LINE_TYPE, false);
/// draw all feature points in the color of the corresponding object for (std::vector
::size_type j = 0; j != trackedObjects[i].getPoints().size(); j++) { // handle weight of point int w = trackedObjects[i].getWeight()[j]; int size = (int)trackedObjects[i].getPoints().size(); if (w != -1) { // point not found // draw filled circle for new points if (w == 1) w = -1; else if (w >= HEATMAP_RANGE) w = HEATMAP_RANGE;
if (w != -1) circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, getHeatMapColor(1 - (float)w / (float)HEATMAP_RANGE), -1, 8); circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, trackedObjects[i].getColor(), 2, 8); } } } } }
Scalar getHeatMapColor(float value) { float red, green, blue; const int NUM_COLORS = 4; static float color[NUM_COLORS][3] = { { 0, 0, 1 }, { 0, 1, 0 }, { 1, 1, 0 }, { 1, 0, 0 } }; // A static array of 4 colors: (blue, green, yellow, red) using {r,g,b} for each.
int idx1; // |-- Our desired color will be between these two indexes in "color". int idx2; // | float fractBetween = 0; // Fraction between "idx1" and "idx2" where our value is.
if (value <= 0) { idx1 = idx2 = 0; } // accounts for an input <=0 else if (value >= 1) { idx1 = idx2 = NUM_COLORS - 1; } // accounts for an input >=0 else { value = value * (NUM_COLORS - 1); idx1 = (int)floor(value); // Our desired color will be after this index. idx2 = idx1 + 1; // ... and before this index (inclusive). fractBetween = value - float(idx1); // Distance between the two indexes (0-1). }
red = ((color[idx2][0] - color[idx1][0])fractBetween + color[idx1][0]) 255; green = ((color[idx2][1] - color[idx1][1])*fractBetween + color[idx1][1])
- 255; blue = ((color[idx2][2] - color[idx1][2])fractBetween + color[idx1][2]) 255;
return Scalar(red, green, blue);
}
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Smorodov/Multitarget-tracker/issues/43#issuecomment-324215822, or mute the thread https://github.com/notifications/unsubscribe-auth/ABu3jdDAJ3O_HP4btBfv1NveFOQHM5plks5sa6YhgaJpZM4O-EOi .
Sir,I need to track the moving objects specially animals,Do you have static background animal moving videos which are more supportive for tracking ,could you send me please sir
On Wednesday, August 23, 2017 3:12 PM, Andrey Smorodov <notifications@github.com> wrote:
Hi, still not clear what do you want, this code is for MOG2, it uses MOG2, and if it works, than no bugs here.
2017-08-23 7:10 GMT+03:00 dulithaaaa notifications@github.com:
Thank you very much sir,sir i have code can you find what is bug in the code please,it works only to MOG2
On Wednesday, August 23, 2017 9:16 AM, Sergey Nuzhny < notifications@github.com> wrote:
Sir,I have a code of tracking i attached that file.it worked only to MOG2,could you please look at that code and tell me what modification needs to be required. Hi! I don't know what do you want. How can I advise you? This project - Multitarget-tracker contains full stack of algorithms for motion detection. I can tell you how it works.
- You need CMake gui application, start it.
- Select folder Mulitarget-tracker (with CMakeLists.txt file).
- Select subfolder folder build.
- Push a button Configure, select your MSVC compler. Wait.
- If OpenCV wasn't founded automatic then select OpenCV_DIR manually. Push a button Configure again.
- Push a button Generate.
- Now the folder build contains a solution file (.sln). Lets open it in Visual studio! The end. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.
include "opencv2/imgcodecs.hpp"
include "opencv2/imgproc.hpp"
include "opencv2/videoio.hpp"
include <opencv2/highgui.hpp>
include <opencv2/video.hpp>
include
include
include
include
include "TrackedObject.h"
using namespace cv; using namespace std;
// Global constant
const bool SHOW_ALL_OPERATION_RESULTS = true; // defines if different operation results should be shown const bool WINDOW_NAMES_READABLE = false; // if set, creates y-offset to make window names readable const int ELEMENTS_PER_ROW = 3; // defines how much windows are shown within a row const bool TRACKING_RESULTS_IN_ORIGINAL = false; // shows tracking results in original frame
// video related constants const int VIDEO_NUMBER = 1; // 1:10, choose different number for different video const bool LOOP_VIDEO = true; // defines if video should be looped infinitely
const char* VIDEO_DIRECTORY_PATH = ".\detect.avi\"; // define path to video directory
const bool SOURCE2GRAY = false; // defines, if source frame should be converted to gray
// foreground segmentation postprocessing related constants const int HISTORY = 1000; // MOG2 parameter: amount of considered frames const double VAR_THRESHOLD = 20; // MOG2 parameter: threshold for degree of minimum variance const bool DETECT_SHADOWS = false; // MOG2 parameter: defines if shadows should be detected
int thresholdMask = 100; // defines start value of threshold for // binarization of foreground frame const int THRESHOLD_MASK_MAX = 255; // defines maximum value for threshold
// morphing related constants const int KERNEL_TYPE = MORPH_ELLIPSE; // choose: MORPH_RECT, MORPH_CROSS, MORPH_ELLIPSE int kernelSizeClose = 5; // defines start value for kernel size of close op. int kernelSizeErode = 5; // defines start value for kernel size of erode op. int kernelSizeDilate = 5; // defines start value for kernel size of dilate op. const int KERNEL_SIZE_MAX = 20; // defines maximum value for threshold
// drawing related constants const Scalar DEFAULT_DRAWING_COLOR = Scalar(0, 255, 0); // defines default color of bounding boxes and labels const int CONTOUR_MIN_AREA = 500; // defines minimum area of contours const int BBOX_MIN_AREA = 3000; // defines minimum area of bounding boxes const bool SHOW_LABELS = false; // shows labels of object adjusted to the bounding box const int HEATMAP_RANGE = 25; // defines size of different colors in heat map // and threshold (color of highest weight)
// bounding box and label related constants const float ALPHA = 0.5; // defines opacity of rectangle and label const int LABEL_HEIGHT = 20; // defines height of label const char* TEST_NAME = "Car"; // defines name for test labels
const int TEXT_FONT_FACE = 2; // defines font of label text const int TEXT_THICKNESS = 2; // defines thickness of label text const int TEXT_LINE_TYPE = 0; // defines line type of label text const double TEXT_FONT_SCALE = 0.7; // defines scale of label text
// tracking related constants const bool TRACKING = true; // defines if tracking is active const bool STATEFUL_TRACKING = true; // defines if stateful tracking with TrackedObjects is active const int MAX_COUNT_TOTAL = 150; // defines max no. of tracked points in total const int MAX_COUNT_PER_CAR = 25; // defines max no. of tracked points per car const int MIN_POINTS_TRACKED_OBJECT = 3; // defines min no. of points for tracked objects const int INIT_INTERVAL = 15; // defines, after how many frames the tracker should initialize new points const TermCriteria termcrit(TermCriteria::COUNT | TermCriteria::EPS, 20, 0.03); // defines criteria for klt tracker const Size subPixWinSize(10, 10); const Size winSize(31, 31);
/// Global variables
// window related variables int scale = 10; // scales windows (and mats) after processing before displaying // (10 is equal to standard size, 20 to doubled size)
// video related variables int frameStart = 1; // defines frame to start (set in video chooser) int frameEnd = 0; // defines frame to end (set in video chooser)
// frame related variables
Mat frame; //current frame Mat fgMaskMOG2; //fg mask fg mask generated by MOG2 method Mat frameGray; // current frame in grayscale Mat frameGrayPrev; // previous frame in grayscale Mat frameThreshold; // frame with applied threshold (to elimante shadows etc.) Mat frameMorphed; // morphed frame (to elimante noise) Mat frameMasked; // morphed frame applied as a mask to the original frame Mat frameContours; // drawn contours Mat frameKltTracking; // klt tracking lanes
Ptr
pMOG2; //MOG2 Background subtractor // tracking related variables bool resetFeatures = false; Point2f point; vector
points[2]; int amountHandledContoursPrev = 0; vector
trackedObjects; int objectCounter; int keyboard; //input from keyboard
/// Function Headers
char* videoChooser();
void processVideo(char* videoFilename);
void showFrame(char* windowName, Mat frame, int width,int height, int position);
void resizeFrames();
void drawBoundingBoxes(vector<vector
> contours, Mat frame); void showControlPanel();
void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector
> contours); void trackObjects(vector<vector
> contours, size_t currentPoint); size_t* getCorrespondingObject(vector
contour); void handleProcessedPoints();
Scalar getHeatMapColor(float value);
void processVideo(char videoFilename); void processImages(char firstFrameFilename);
char* VIDEO_FILE = "visiontraffic.avi";
int main(int argc, char* argv[]) {
char* videoPath = new char[100];
/// initialize variables // define path of video file strcpy(videoPath, VIDEO_DIRECTORY_PATH); strcat(videoPath, videoChooser());
//create GUI windows namedWindow("Original"); namedWindow("Foreground (MOG2)"); namedWindow("Threshold"); namedWindow("Morphed"); namedWindow("Contours"); namedWindow("Masked Original Frame"); namedWindow("KLT Tracking");
//create Background Subtractor objects pMOG2 = createBackgroundSubtractorMOG2(HISTORY, VAR_THRESHOLD, DETECT_SHADOWS); //MOG2 approach
//input data coming from a video processVideo(VIDEO_FILE); //destroy GUI windows destroyAllWindows(); return EXIT_SUCCESS; }
void processVideo(char* videoFilename) {
//create the capture object VideoCapture capture(videoFilename); if (!capture.isOpened()) { //error in opening the video input cerr << "Unable to open video file: " << videoFilename << endl; exit(EXIT_FAILURE); }
///read input data. ESC or 'q' for quitting while ((char)keyboard != 'q' && (char)keyboard != 27) { //read the current frame if (!capture.read(frame)) { cerr << "Unable to read next frame." << endl; cerr << "Exiting..." << endl; exit(EXIT_FAILURE); }
//update the background model pMOG2->apply(frame, fgMaskMOG2);
stringstream ss; rectangle(frame, cv::Point(10, 2), cv::Point(100, 20), cv::Scalar(255, 255, 255), -1); ss << capture.get(CAP_PROP_POS_FRAMES); string frameNumberString = ss.str(); putText(frame, frameNumberString.c_str(), cv::Point(15, 15), FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0, 0));
imshow("Frame", frame); imshow("FG Mask MOG 2", fgMaskMOG2);
pMOG2->apply(frame, fgMaskMOG2);
imshow("Frame", frame); imshow("FG Mask MOG 2", fgMaskMOG2);
//get the frame number and write it on the current frame rectangle(frame, Point(10, 2), Point(100, 20), Scalar(255, 255, 255), -1); frameNumberCurrent << capture.get(CAP_PROP_POS_FRAMES); frameInfo << frameNumberCurrent.str() << "/" << frameNumberTotal.str(); putText(frame, frameInfo.str(), Point(15, 15), FONT_HERSHEY_SIMPLEX, 0.5, Scalar(0, 0, 0));
//get the input from the keyboard keyboard = waitKey(30);
/// variable declaration and initialization // morphing related variables (initialized during each loop, since parameters are changable) Mat kernelClose; Mat kernelErode; Mat kernelDilate;
// frame related variables stringstream frameNumberTotal; stringstream frameNumberCurrent; stringstream frameInfo; int frameWidth = 0, frameHeight = 0;
// contour related variables vector<vector
> contours; vector hierarchy; // displaying related variables int r = 1; // counter for row
//// video processing
frameNumberTotal << capture.get(CAP_PROP_FRAME_COUNT); frameWidth = (int)capture.get(CV_CAP_PROP_FRAME_WIDTH); frameHeight = (int)capture.get(CV_CAP_PROP_FRAME_HEIGHT);
if (frameEnd == 0) frameEnd = (int)capture.get(CAP_PROP_FRAME_COUNT);
// set video to defined start frame capture.set(CAP_PROP_POS_FRAMES, frameStart);
// show control panel (enables modification of some variables in realtime) showControlPanel();
while ((char)keyboard != 27) { // read the current frame if (!capture.read(frame)) { cerr << "Unable to read next frame." << endl; cerr << "Exiting..." << endl; exit(EXIT_FAILURE); }
// convert original frame to grayscale if specified by constant if (SOURCE2GRAY) frameGray.copyTo(frame);
// re-initialize objects with changable parameters if (kernelSizeClose > 0) kernelClose = getStructuringElement(KERNEL_TYPE, Size(kernelSizeClose, kernelSizeClose)); if (kernelSizeErode > 0) kernelErode = getStructuringElement(KERNEL_TYPE, Size(kernelSizeErode, kernelSizeErode)); if (kernelSizeDilate > 0) kernelDilate = getStructuringElement(KERNEL_TYPE, Size(kernelSizeDilate, kernelSizeDilate));
} // apply threshold to MOG2 frame to get rid of shadows etc threshold(fgMaskMOG2, frameThreshold, thresholdMask, 255, THRESH_BINARY);
// apply morphology operations to thresholded frame
if (kernelSizeClose > 0) morphologyEx(frameMorphed, frameMorphed, MORPH_CLOSE, kernelClose); if (kernelSizeErode > 0) morphologyEx(frameMorphed, frameMorphed, MORPH_ERODE, kernelErode); if (kernelSizeDilate > 0) morphologyEx(frameMorphed, frameMorphed, MORPH_DILATE, kernelDilate);
frameThreshold.copyTo(frameMorphed);
// save previous masked frame as grayscale frame for later tracking if (!frameMasked.empty()) cvtColor(frameMasked, frameGrayPrev, CV_RGB2GRAY);
// apply morphed mask, but save before the morphed frame before frameMorphed.copyTo(frameContours); frameMorphed.copyTo(frameMasked); frame.copyTo(frameMasked, frameMasked);
// save new masked frame as grayscale frame for later tracking, also save mask to KltTracking frame cvtColor(frameMasked, frameGray, CV_RGB2GRAY); frameMasked.copyTo(frameKltTracking);
// draw and fill contours findContours(frameContours, contours, hierarchy, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE); setIdentity(frameContours, Scalar(0, 0, 0)); drawContours(frameContours, contours, -1, Scalar(255, 255, 255), -1);
// draws rectangle around identified contours drawBoundingBoxes(contours, frameMasked);
// prepare for and apply tracking algorithm (use grayscale copies of masked frame) if (!frameGrayPrev.empty() && TRACKING) applyKltTracking(frameGray, frameGrayPrev, frameKltTracking, contours);
/// show the different frames (from last to first row, to overlay windows bar) resizeFrames(); // resize all frames
if (SHOW_ALL_OPERATION_RESULTS) { // row 3 r = 3; showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW);
// row 2 r = 2; showFrame("Morphed", frameMorphed, frameWidth, frameHeight, 1 + (r - 1)ELEMENTS_PER_ROW); showFrame("Contours", frameContours, frameWidth, frameHeight, 2 + (r - 1)ELEMENTS_PER_ROW); showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 3
- (r - 1)*ELEMENTS_PER_ROW);
// row 1 r = 1; showFrame("Original", frame, frameWidth, frameHeight, 1 + (r - 1)*ELEMENTS_PER_ROW); showFrame("Foreground (MOG2)", fgMaskMOG2, frameWidth, frameHeight, 2 + (r
- 1)ELEMENTS_PER_ROW); showFrame("Threshold", frameThreshold, frameWidth, frameHeight, 3 + (r - 1)ELEMENTS_PER_ROW);
} else { // row 1 r = 1; showFrame("Masked Original Frame", frameMasked, frameWidth, frameHeight, 1
- (r - 1)ELEMENTS_PER_ROW); showFrame("KLT Tracking", frameKltTracking, frameWidth, frameHeight, 2 + (r - 1)ELEMENTS_PER_ROW); }
// shows image for the specified amount of milliseconds keyboard = waitKey(30);
switch (keyboard) { case 'r': resetFeatures = true; // reset on pressed 'r' break; }
// loop through the video if end is reached and loop is set if (LOOP_VIDEO && (frameNumberCurrent.str().compare(std::to_string(frameEnd)) == 0)) capture.set(CAP_PROP_POS_FRAMES, frameStart);
// ensure that new feature points are initialized every INIT_INTERVAL frames for the tracker if (std::stoi(frameNumberCurrent.str()) % INIT_INTERVAL == 0 && trackedObjects.size() > 0) resetFeatures = true;
// cleaning operation: clean string streams frameNumberCurrent.str(""); frameInfo.str(""); }
// cleaning operation: release capture object capture.release();
//imshow("FG Mask MOG 2", fgMaskMOG2);
//get the input from the keyboard //keyboard = waitKey(30); //} //delete capture object ////capture.release(); }
char* videoChooser() {
// variable declaration char* videoFile = "dog.avi"; // saves name of selected video file
// chooses video file depending on chosen video number switch (VIDEO_NUMBER) { case 1: videoFile = "visiontraffic.avi"; frameStart = 90; frameEnd = 440; break;
case 2: videoFile = "viptraffic.avi"; break;
case 3: videoFile = "highwayII_raw.avi"; break;
case 4: videoFile = "dispersed.ogv"; break;
case 5: videoFile = "highspeed.ogv"; break;
case 6: videoFile = "hightraffic-dark.ogv"; break;
case 7: videoFile = "hightraffic-occlude.ogv"; break;
case 8: videoFile = "threecars.ogv"; break;
case 9: videoFile = "turntraffic-dark.ogv"; break;
case 10: videoFile = "visor_1261565143617_cesta1.wmv"; break;
default: break; }
return videoFile; }
void showFrame(char* windowName, Mat frame, int width, int height, int position) { // variable declaration and initialization int x = 0, // x-coordinates for position y = 0, // y-coordinates for position yOffset = 30; // offset for y-coordinates, since windows bar does not count into frame height // strategy for y-offset: create windows from last row to first to overlay the bar
// create y-Offset if defince to make titles of windows readable if (!WINDOW_NAMES_READABLE) { yOffset += -32; } else if (position > ELEMENTS_PER_ROW) { yOffset += 32; }
// create window and display defined frame
cv::namedWindow(windowName); cv::imshow(windowName, frame);
// resize stats for position calculation, if defined if (scale != 10) { // filter for the case, that it's zero, since range is not adjustable in opencv gui if (scale == 0) scale = 1;
// calculate scalefactor from value of scale variable float scaleFactor = (float)scale / 10;
// calculate new width and height width = (int)(width scaleFactor); height = (int)(height scaleFactor); }
//// get positioning
// get x coordinates with width if (((position - 1) % ELEMENTS_PER_ROW) != 0) x = ((position - 1) % ELEMENTS_PER_ROW) * width;
// get y coordinates with height if (position > ELEMENTS_PER_ROW) y = (position - (position - 1) % ELEMENTS_PER_ROW) / ELEMENTS_PER_ROW * height;
// place window at calculated position moveWindow(windowName, x, y + yOffset); }
void resizeFrames() { //resize all frames, if defined if (scale != 10 && scale > 0) { // filter for the case, that it's zero, since range is not adjustable in opencv gui if (scale == 0) scale = 1;
// calculate scalefactor from value of scale variable float scaleFactor = (float)scale / 10;
// resize all frames resize(frame, frame, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(fgMaskMOG2, fgMaskMOG2, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameThreshold, frameThreshold, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameMorphed, frameMorphed, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameMasked, frameMasked, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameContours, frameContours, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); resize(frameKltTracking, frameKltTracking, Size(100,100), scaleFactor, scaleFactor, CV_INTER_LINEAR); } }
void drawBoundingBoxes(vector<vector
> contours, Mat frame) { // variable declaration int amountHandledContours = 0; Scalar drawingColor = DEFAULT_DRAWING_COLOR; // create bounding rect of object and draw on defined frame for (vector<vector
>::iterator itc = contours.begin(); itc != contours.end(); itc++) { // variable declaration and initialization Rect boundingBox = boundingRect(Mat(*itc)); Mat overlay; // only handle rectangles with a minimum area if (boundingBox.area() >= CONTOUR_MIN_AREA) { if (contourArea(*itc) >= CONTOUR_MIN_AREA) {
// get corresponding object, if there's one, to define the drawing color size_t indexCorrespondingObject = getCorrespondingObject(itc); if (indexCorrespondingObject != NULL) drawingColor = trackedObjects[*indexCorrespondingObject].getColor();
// draw contours regardless of area drawContours(frame, contours, -1, drawingColor, 1);
// handle bounding boxes (but only if they're bigger than defined minimum area) if (boundingBox.area() >= BBOX_MIN_AREA) { // copy frame to frame.copyTo(overlay);
// draw rectangle rectangle(overlay, boundingBox, drawingColor, 3);
// draw test label if constant is set if (SHOW_LABELS) rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y), Point(boundingBox.x + boundingBox.width + 1, boundingBox.y - LABEL_HEIGHT), drawingColor, -1);
// combine transparent overlay of bounding rect with original frame cv::addWeighted(overlay, ALPHA, frame, 1 - ALPHA, 0, frame);
// afterwards, put text with object name into the label, if constant is set if (SHOW_LABELS) putText(frame, TEST_NAME, Point(boundingBox.x + 10, boundingBox.y - 2), TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(0, 0, 0), TEXT_THICKNESS, TEXT_LINE_TYPE, false);
// count handled contours amountHandledContours++; } } } }
// re-initialize feature tracker if new objects appeared in this frame if (amountHandledContours > amountHandledContoursPrev) { resetFeatures = true; cout << amountHandledContours << "/" << amountHandledContoursPrev << endl; }
amountHandledContoursPrev = amountHandledContours; }
void showControlPanel() { // constant declaration const int WIDTH = 400, HEIGHT = 240;
const char nameWindow = "Control Panel"; const char nameTrackbarWindowScale = "Scale"; const char nameTrackbarThreshold = "Threshold"; const char nameTrackbarKernelSizeClose = "SizeClose"; const char nameTrackbarKernelSizeErode = "SizeErode"; const char nameTrackbarKernelSizeDilate = "SizeDilate";
// create window namedWindow(nameWindow, CV_WINDOW_AUTOSIZE | CV_GUI_NORMAL); resizeWindow(nameWindow, WIDTH, HEIGHT);
// window relating parameters createTrackbar(nameTrackbarWindowScale, nameWindow, &scale, 20, NULL); createTrackbar(nameTrackbarThreshold, nameWindow, &thresholdMask, THRESHOLD_MASK_MAX, NULL);
createTrackbar(nameTrackbarKernelSizeClose, nameWindow, &kernelSizeClose, KERNEL_SIZE_MAX, NULL); createTrackbar(nameTrackbarKernelSizeErode, nameWindow, &kernelSizeErode, KERNEL_SIZE_MAX, NULL); createTrackbar(nameTrackbarKernelSizeDilate, nameWindow, &kernelSizeDilate, KERNEL_SIZE_MAX, NULL); }
/**
- @function for klt tracking */ void applyKltTracking(Mat frameGray, Mat frameGrayPrev, Mat frameKltTracking, vector<vector
> contours) { /// variable declaration
vector
newPoints; /// calculate features and then the optical flow if (resetFeatures) { // automatic initialization goodFeaturesToTrack(frameGray, newPoints, MAX_COUNT_TOTAL, 0.01, 10, Mat(), 3, 0, 0.04); cornerSubPix(frameGray, newPoints, subPixWinSize, Size(-1, -1), termcrit);
//points[0].clear(); // could solve performance issue, but creates strange behaviour: // trackedObjects also reset every reset //points[1].clear();
// initialize with all available points from tracked objects for (std::vector
::size_type l = 0; l != trackedObjects.size(); l++) for (std::vector ::size_type j = 0; j != trackedObjects[l].getPoints().size(); j++) { points[0].push_back(trackedObjects[l].getPoints()[j]); } } if (!points[0].empty() || trackedObjects.size() > 0) { // variable declaration vector status; vector err; size_t i, k; // check if previous frame is existing, otherwise copy the current one if (frameGrayPrev.empty()) frameGray.copyTo(frameGrayPrev);
// calculate the optical flow with KLT calcOpticalFlowPyrLK(frameGrayPrev, frameGray, points[0], points[1], status, err, winSize, 3, termcrit, 0, 0.001);
/// iterate through all found points for (i = k = 0; i < points[1].size(); i++) {
// don't handle point, if there's no corresponding point from the last frame if (!status[i]) continue;
// otherwise save and draw point (shrink's point vector, resizes after the for loop) points[1][k++] = points[1][i];
/// get corresponding contour and object, otherwise create new tracked object if (STATEFUL_TRACKING) trackObjects(contours, i); else circle(frameKltTracking, points[1][i], 6, DEFAULT_DRAWING_COLOR, 2, 8); }
points[1].resize(k); }
// handle the processed points handleProcessedPoints();
// save that initialization was done and not needed anymore, also add new points if (resetFeatures) { resetFeatures = false;
for (std::vector
::size_type l = 0; l != newPoints.size(); l++) points[1].push_back(newPoints[l]); } // save the current values as the previous ones if (!resetFeatures) std::swap(points[1], points[0]);
cv::swap(frameGrayPrev, frameGrayPrev); }
void trackObjects(vector<vector
> contours, size_t currentPoint) { /// variable declaration int containedInContour; bool foundObject = false; bool foundContour = false; bool foundObjectOfContour = false; vector ::size_type m; // iterator for contours size_t* correspondingObject = NULL; // pointer for corresponding object /// get contour of point for (m = 0; m != contours.size(); m++) { // checks if point is inside or on the edge of the contour containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false);
// exit loop if corresponding contour is found if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) { foundContour = true; break; } }
if (trackedObjects.size() > 0) { // check for each point in the tracked object if it already exists for (std::vector
::size_type l = 0; l != trackedObjects.size(); l++) { size_t* containedInObject = NULL; if (trackedObjects[l].containsOldPoint(points[0][currentPoint]) != NULL) { // add the point (it will increase weight automatically instead) trackedObjects[l].addPoint(points[1][currentPoint], points[0][currentPoint]); foundObject = true;
// update contour of object, if contour was not modified this frame yet bool foundContour = false; if (!trackedObjects[l].getContourStatus()) { std::vector
::size_type m; for (m = 0; m != contours.size(); m++) { // checks if point is inside or on the edge of the contour containedInContour = (int)pointPolygonTest(contours[m], points[1][currentPoint], false); // exit loop if corresponding contour is found if (containedInContour > 0 && contourArea(contours[m]) >= CONTOUR_MIN_AREA) { foundContour = true; break; } }
// update corresponding contour, if one was found if (foundContour) trackedObjects[l].setContour(contours[m]); } }
} }
// create new object with this point if (!foundObject && foundContour) { size_t* containedInObject; containedInObject = getCorrespondingObject(contours[m]);
if (containedInObject != NULL) { // contour belongs to an existing object, so add the point size_t containedInObjectVal = *containedInObject;
// add points if (trackedObjects[containedInObjectVal].getPoints().size() < MAX_COUNT_PER_CAR) trackedObjects[containedInObjectVal].addPoint(points[1][currentPoint], points[0][currentPoint]);
foundObjectOfContour = true; }
// create new tracked object, since there's no existing one with this contour if (!foundObjectOfContour) { char name[11] = "Object_"; char objectNumber[4]; sprintf(objectNumber, "%d", (int)(objectCounter + 1)); strcat(name, objectNumber);
TrackedObject* trObj = new TrackedObject(name); trObj->addPoint(points[1][currentPoint], points[0][currentPoint]);
trObj->setContour(contours[m]);
trackedObjects.push_back(*trObj); objectCounter++; } } }
size_t getCorrespondingObject(vector
contour) { // variable declaration size_t position = NULL;// find out if contour is assigned to existing object if (trackedObjects.size() > 0 && contour.size() > 0) { for (std::vector
::size_type i = 0; i != trackedObjects.size(); i++) { if (trackedObjects[i].getContour().size() == contour.size()) { double comparisonValue; comparisonValue = matchShapes(trackedObjects[i].getContour(), contour, CV_CONTOURS_MATCH_I1, NULL); if (comparisonValue == 0) { position = &i; break; } } } } return position; }
void handleProcessedPoints() { // get original frame, if constant defines, that results should be shown in original if (TRACKING_RESULTS_IN_ORIGINAL) frame.copyTo(frameKltTracking);
if (trackedObjects.size() > 0) { for (std::vector
::size_type i = 0; i != trackedObjects.size(); i++) { // process all points according to the status trackedObjects[i].processStatus();
if (trackedObjects[i].getPoints().size() <= MIN_POINTS_TRACKED_OBJECT) { trackedObjects.erase(trackedObjects.begin() + i); i--; continue; }
/// draw bounding box Rect boundingBox = boundingRect(trackedObjects[i].getContour()); Mat overlay; frameKltTracking.copyTo(overlay);
/// draw rectangle and label rectangle(overlay, boundingBox, trackedObjects[i].getColor(), 3); rectangle(overlay, Point(boundingBox.x - 2, boundingBox.y), Point(boundingBox.x + boundingBox.width + 1, boundingBox.y - LABEL_HEIGHT), trackedObjects[i].getColor(), -1);
// combine transparent overlay of bounding rect with original frame cv::addWeighted(overlay, ALPHA, frameKltTracking, 1 - ALPHA, 0, frameKltTracking);
// afterwards, put text with object name into the label, if constant is set putText(frameKltTracking, trackedObjects[i].getName(), Point(boundingBox.x
- 10, boundingBox.y - 2), TEXT_FONT_FACE, TEXT_FONT_SCALE, Scalar(200, 200, 200), TEXT_THICKNESS, TEXT_LINE_TYPE, false);
/// draw all feature points in the color of the corresponding object for (std::vector
::size_type j = 0; j != trackedObjects[i].getPoints().size(); j++) { // handle weight of point int w = trackedObjects[i].getWeight()[j]; int size = (int)trackedObjects[i].getPoints().size(); if (w != -1) { // point not found // draw filled circle for new points if (w == 1) w = -1; else if (w >= HEATMAP_RANGE) w = HEATMAP_RANGE;
if (w != -1) circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, getHeatMapColor(1 - (float)w / (float)HEATMAP_RANGE), -1, 8); circle(frameKltTracking, trackedObjects[i].getPoints()[j], 6, trackedObjects[i].getColor(), 2, 8); } } } } }
Scalar getHeatMapColor(float value) { float red, green, blue; const int NUM_COLORS = 4; static float color[NUM_COLORS][3] = { { 0, 0, 1 }, { 0, 1, 0 }, { 1, 1, 0 }, { 1, 0, 0 } }; // A static array of 4 colors: (blue, green, yellow, red) using {r,g,b} for each.
int idx1; // |-- Our desired color will be between these two indexes in "color". int idx2; // | float fractBetween = 0; // Fraction between "idx1" and "idx2" where our value is.
if (value <= 0) { idx1 = idx2 = 0; } // accounts for an input <=0 else if (value >= 1) { idx1 = idx2 = NUM_COLORS - 1; } // accounts for an input >=0 else { value = value * (NUM_COLORS - 1); idx1 = (int)floor(value); // Our desired color will be after this index. idx2 = idx1 + 1; // ... and before this index (inclusive). fractBetween = value - float(idx1); // Distance between the two indexes (0-1). }
red = ((color[idx2][0] - color[idx1][0])fractBetween + color[idx1][0]) 255; green = ((color[idx2][1] - color[idx1][1])*fractBetween + color[idx1][1])
- 255; blue = ((color[idx2][2] - color[idx1][2])fractBetween + color[idx1][2]) 255;
return Scalar(red, green, blue);
}
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Smorodov/Multitarget-tracker/issues/43#issuecomment-324215822, or mute the thread https://github.com/notifications/unsubscribe-auth/ABu3jdDAJ3O_HP4btBfv1NveFOQHM5plks5sa6YhgaJpZM4O-EOi .
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.
No I have no such videos.
Sir,I try to run your code.But i have not clear idea about which files needs to be added.It means which header and source files required.could you please tell me how to run this project correctly.