opencv3 class TrackerSamplerPF ERROR

[shodow]

When i using class TrackerSamplerPF,i get this error: Error: Assertion failed (0 <= _rowRange.start && _rowRange.start <= _rowRange.end && _rowRange.end <= m.rows) this is my code:

#include "opencv2/opencv.hpp"
#include "opencv2/tracking.hpp"

using namespace cv;
using namespace std;
int main()
{
    VideoCapture video("2.avi");

    // Exit if video is not opened
    if (!video.isOpened())
    {
        cout << "Could not read video file" << endl;
        return 1;
    }

    // Read first frame 
    Mat frame;
    bool ok = video.read(frame);

    // Define initial bounding box 
    Rect bbox;
    vector<Mat> sample;
    // Uncomment the line below to select a different bounding box 
    bbox = selectROI(frame, false);
    // Display bounding box. 
    rectangle(frame, bbox, Scalar(255, 0, 0), 2, 1);
    imshow("Tracking", frame);

    vector<Mat> sampleImg;
    Mat roi = frame(bbox);
    TrackerSamplerPF *PFSampler = new TrackerSamplerPF(roi);
    cout << PFSampler->getClassName() << endl;

    while (1)
    {
        Mat new_frame;
        video >> new_frame;

        ok = PFSampler->sampling(new_frame, bbox, sampleImg);
        //bool ok = true;
        if (ok)
        {
            //Tracking success : Draw the tracked object
            for (auto elem : sampleImg)
            {
                imshow("Tracking1", elem);
            }
            rectangle(new_frame, bbox, Scalar(255, 0, 0), 2, 1);
        }
        else
        {
            //Tracking failure detected.
            putText(new_frame, "Tracking failure detected", Point(100, 80), FONT_HERSHEY_SIMPLEX, 0.75, Scalar(0, 0, 255), 2);
        }

        imshow("Tracking", new_frame);

        //Exit if ESC pressed.
        int k = waitKey(33);
        if (k == 27)
        {
            break;
        }
    }

    return 1;
}

[alalek]

Minimal complete reproducer is required (including input data).

[shodow]

@alalek I'm sorry I don't understand what's you mean :(

[berak]

@shodow, can it be reproduced with the video from here and the 1st bounding box from here ?

(we don't know about your video, and where you clicked in there, so it's better to use something "known")

[shodow]

@berak hi, i use this video and get the same error. the 1st bounding box is 123,87,132,176

[alalek]

OpenCV version? (cv::getBuildInformation()) Did you try latest code from 3.4 branch?

[shodow]

@alalek this function(cv::getBuildInformation()) return

``` General configuration for OpenCV 3.4.3-dev ===================================== Version control: unknown Extra modules: Location (extra): D:/come on/opencv_contrib-3.4/modules Version control (extra): unknown Platform: Timestamp: 2018-10-19T02:51:13Z Host: Windows 10.0.17134 AMD64 CMake: 3.12.2 CMake generator: Visual Studio 15 2017 CMake build tool: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/MSBuild/15.0/Bin/MSBuild.exe MSVC: 1915 CPU/HW features: Baseline: SSE SSE2 requested: SSE2 Dispatched code generation: SSE4_1 SSE4_2 FP16 AVX requested: SSE4_1 SSE4_2 AVX FP16 SSE4_1 (6 files): + SSE3 SSSE3 SSE4_1 SSE4_2 (2 files): + SSE3 SSSE3 SSE4_1 POPCNT SSE4_2 FP16 (1 files): + SSE3 SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX AVX (6 files): + SSE3 SSSE3 SSE4_1 POPCNT SSE4_2 AVX C/C++: Built as dynamic libs?: YES C++11: YES C++ Compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.15.26726/bin/Hostx86/x86/cl.exe (ver 19.15.26729.0) C++ flags (Release): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast /arch:SSE /arch:SSE2 /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP8 /MD /O2 /Ob2 /DNDEBUG C++ flags (Debug): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast /arch:SSE /arch:SSE2 /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP8 /MDd /Zi /Ob0 /Od /RTC1 C Compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.15.26726/bin/Hostx86/x86/cl.exe C flags (Release): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast /arch:SSE /arch:SSE2 /MP8 /MD /O2 /Ob2 /DNDEBUG C flags (Debug): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /fp:fast /arch:SSE /arch:SSE2 /MP8 /MDd /Zi /Ob0 /Od /RTC1 Linker flags (Release): /machine:X86 /INCREMENTAL:NO Linker flags (Debug): /machine:X86 /debug /INCREMENTAL ccache: NO Precompiled headers: YES Extra dependencies: 3rdparty dependencies: OpenCV modules: To be built: aruco bgsegm bioinspired calib3d ccalib core datasets dnn dnn_objdetect dpm face features2d flann fuzzy hdf hfs highgui img_hash imgcodecs imgproc java_bindings_generator line_descriptor ml objdetect optflow phase_unwrapping photo plot python3 python_bindings_generator reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab xfeatures2d ximgproc xobjdetect xphoto Disabled: world Disabled by dependency: - Unavailable: cnn_3dobj cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev cvv freetype java js matlab ovis python2 sfm viz xfeatures2d Applications: tests perf_tests apps Documentation: NO Non-free algorithms: NO Windows RT support: NO GUI: Win32 UI: YES VTK support: NO Media I/O: ZLib: build (ver 1.2.11) JPEG: build-libjpeg-turbo (ver 1.5.3-62) WEBP: build (ver encoder: 0x020e) PNG: build (ver 1.6.34) TIFF: build (ver 42 - 4.0.9) JPEG 2000: build (ver 1.900.1) OpenEXR: build (ver 1.7.1) HDR: YES SUNRASTER: YES PXM: YES Video I/O: Video for Windows: YES DC1394: NO FFMPEG: YES (prebuilt binaries) avcodec: YES (ver 57.107.100) avformat: YES (ver 57.83.100) avutil: YES (ver 55.78.100) swscale: YES (ver 4.8.100) avresample: YES (ver 3.7.0) GStreamer: NO DirectShow: YES Media Foundation: YES Parallel framework: Concurrency Trace: YES (with Intel ITT) Other third-party libraries: Intel IPP: 2017.0.3 [2017.0.3] at: D:/come on/opencv_contrib_x86/3rdparty/ippicv/ippicv_win Intel IPP IW: sources (2017.0.3) at: D:/come on/opencv_contrib_x86/3rdparty/ippicv/ippiw_win Lapack: NO Eigen: NO Custom HAL: NO Protobuf: build (3.5.1) OpenCL: YES (no extra features) Include path: D:/come on/opencv-3.4/3rdparty/include/opencl/1.2 Link libraries: Dynamic load Python 3: Interpreter: E:/Program Files/anaconda/python.exe (ver 3.6.5) Libraries: E:/Program Files/anaconda/libs/python36.lib (ver 3.6.5) numpy: E:/Program Files/anaconda/lib/site-packages/numpy/core/include (ver 1.14.3) packages path: E:/Program Files/anaconda/Lib/site-packages Python (for build): E:/Program Files/anaconda/python.exe Java: ant: NO JNI: NO Java wrappers: NO Java tests: NO Install to: D:/come on/opencv_contrib_x86/install ----------------------------------------------------------------- ```

[berak]

i could reproduce it on master:

Thread 1 hit Breakpoint 7, cv::PFSolver::iteration (this=0xb7b92d0) at C:/p/opencv_contrib/modules/tracking/src/PFSolver.hpp:109
109             Mat_<double> maxrow=_particles.row((int)max_element);
(gdb) p (int)max_element
$19 = -4
(gdb) p _particles.rows
$20 = 100

also: https://github.com/opencv/opencv_contrib/blob/91e16573629439029cbe8fabf163406ef071cbda/modules/tracking/src/PFSolver.hpp#L56

it should better use a fixed seed (for reproducability)

[alalek]

Reproduced with:

bbox = Rect(57, 27, 454, 417);

[shodow]

@alalek you mean use this bbox = Rect(57, 27, 454, 417); as 1st bounding box?

[shodow]

@berak thanks for your help,so what should i do?Should I need to modify the source code?

[shodow]

i modify the source code here

Mat_<double> maxrow=_particles.row((int)max_element); to Mat_<double> maxrow=_particles.row((int)-max_element); it can be run,but the sampleImg return is 0,and running very slow

[berak]

@nailbiter, maybe you can shed some light on it ?

@shodow , no i don't think, this is the correct answer. ;)

if _logweight represents a probability, or a loglikelihood for a given particle to be a good seed for the next iteration, then the problem is here:

    double max_element;
    minMaxLoc(_logweight, 0, &max_element);
    Mat_<double> maxrow=_particles.row((int)max_element);

which imho should be:

    Point max_pos;
    minMaxLoc(_logweight, 0, 0, 0, &max_pos);
    Mat_<double> maxrow=_particles.row(max_pos.x);

(there is one weight for each particle, and we want the particle with the strongest weight, so we need the index of it, not the weight itself)

and there might be still more errors lurking in the weights calculation. (the particles look quite ok to me)

btw, we should use a color video, not a grayscale one, for better results here, since it uses HSV histograms to evaluate candidate rects

[nailbiter]

@shodow @berak Guys, I wrote the code a while ago and haven't touched it since then, so it will take me some time. I will take a look and reply within 24 hours.

[fabianstahl]

I receive the same Error as @berak . Did you check the Code? Also, are there any examples on how to use the Particle Filter?

[berak]

@Melodeathguy no, there are no usage examples it's also never used internally, and the "factory" here won't allow to create one.

but i tested the code, and the fix mentioned in the comment above did work for me. (i had do adjust to adjust the pertubation params according to the video size, though, to get something halfway stable, e.g. 15 was far too much for the 320x240 test videos. (it's also very slow, like close to a second per iteration)

just curious, what are you trying to do with it ? (it's not a "standalone" tracker)