OpenCV error during tracking

[lejlot]

I've recently received following error during ManyTrack tracking:

---

::: time parallel remove 0.00151422 ::: time parallel remove 0.000756198 ::: time parallel remove 0.000886443 ::: time parallel remove 0.00119901 ::: ICPTracker processFrame() 1.81022 ::: updateVideoImage() 0.00351012 ::: updateVisualization () 0.0062632 ::: Get updateFrame() 1.20783 OpenCV Error: Sizes of input arguments do not match (The operation is neither 'array > op array' (where arrays have the same size and the same number of channels), nor 'array op scalar', nor 'scalar op array') in arithm_op, file /home/biotracking/Downloads/OpenCV-2.4.3/modules/core/src/arithm.cpp, line 1273 terminate called after throwing an instance of 'cv::Exception' what(): /home/biotracking/Downloads/OpenCV-2.4.3/modules/core/src/arithm.cpp:1273: error: (-209) The operation is neither 'array op > array' (where arrays have the same size and the same number of channels), nor 'array > op scalar', nor 'scalar op array' in function arithm_op

Aborted

Error is repeatable and independent from used parameters (at least for a few checked sets).

[bhrolenok]

Can you provide more details about when the error occurs? I've been testing building/running on a fresh Ubuntu install with a newer version of OpenCV (2.4.6 instead of 2.4.2) and I'm seeing a similar error whenever I try to visualize the background subtracted image.

[bhrolenok]

So I think the error is related to image dimensions (width x height). At least with ModelMaker, when I used a background that was generated with an older version of Backgrounder (before I had updated OpenCV to 2.4.6), I got the error when trying to view the background subtraction, but if I used a Background generated after the update, it worked fine (no crash). Comparing the two images, the older image reports the same dimensions as the video, 1920x1088. The newer image is actually NOT the same as the video dimensions at 1920x1080. This leads me to think there's been a change in either OpenCV or it's video codec libraries that's silently scaling the video when it's loaded, since 1920x1088 is a weird, non-standard video resolution.

This shouldn't be a problem with the Background image, because the background image is checked after it's loaded to make sure it matches the video, and the tracker will not start if they do not match. However, it looks like there's no similar check for the mask image. So I'm guessing that's what's causing the problem

A possible workaround: If you're using a mask image, first use Backgrounder on your source video image to generate a PNG which will have the same dimensions as the scaled video. Then take your hand-made mask image, and re-scale it to match the dimensions of the background video.

Let me know if this works or if it's a completely unrelated bug.

[lejlot]

In my case, tracking error occurs in the middle of the tracking procedure, so it already did compute many things, but fails on some particular frame.

Some information regarding my movie:

avprobe version 0.8.6-4:0.8.6-0ubuntu0.12.04.1, Copyright (c) 2007-2013 the Libav developers built on Apr 2 2013 17:02:36 with gcc 4.6.3 Input #0, avi, from 'test.avi': Metadata: encoder : MEncoder svn r34540 (Ubuntu), built with gcc-4.6 Duration: 00:04:36.26, start: 0.000000, bitrate: 495 kb/s Stream #0.0: Video: mpeg4 (Simple Profile), yuv420p, 1280x720 [PAR 1:1 DAR 16:9], 30 tbr, 1k tbn, 1k tbc

Configuration information (but as I stated before - it occurs also with different settings):

[General] bgsubthresh=100 blobbirth=79 resolutionAnal=8 trackdist=164 trackdeath=30 separation=25 icpmaxiterations=150 icpeuclidean=1000 icpepsilon=0.01 colorRegspinbox=0 icpmaxworstscore=1000 bgpath1=bg.png videopath1=test.avi maskpath1=(none selected) modelpath1=models projpath1= bgsubcheck=true modelshow=true separationview=true

Background information:

bg.png PNG 1280x720 1280x720+0+0 8-bit DirectClass 630KB 0.000u 0:00.000

System information:

OS: Ubuntu 12.04.2 LTS (also fails on OpenSuse) OpenCV: 2.5.0 Code: self compiled

[bhrolenok]

Is it the same frame every time? Or does it appear to be random?

[lejlot]

It seems to appear in the same frame, slightly after the middle so it can be data specific (but movie works fine, backgrounder can read each of its frames etc.).

When it appeared for the first time I concatenated two copies of this movie with itself and runned tracking, and what is very suprising - it went ok through the first half (so whole movie) and crashed in the second one, but this test was performed while ago, with only one set of parameters and testing machine so it may be missleading observation.

[bhrolenok]

Do you always see the error right after that same debug output ("Get updateFrame()")? You can throw a call to capture.get(CV_CAP_PROP_POS_FRAMES); and print it out right after that cout (line 76, Manytrack.cpp) to verify which frame it's dying on.

The fact that it got halfway through the concatenated video (and entirely through the original video) isn't that odd. If the error was an encoding error, you would have re-encoded the file when you made the longer one. Since I can't reproduce this with the videos I have I suspect it's an encoding issue that Backgrounder is handling gracefully where as ManyTrack is throwing an error.

From your output it looks like your video is in an avi container. I'd try removing this first with "ffmpeg -i test.avi -vcodec copy test.mp4", and if you get the same error on the raw mp4 file try re-encoding it with "ffmpeg -i test.avi test.mp4". If none of this works, I can send you a short synthetic video we use for testing that I know works.

[lejlot]

I can send you a short synthetic video we use for testing that I know works.

I have plenty of working videos, there are just some causing this error

I'd try removing this first with "ffmpeg -i test.avi -vcodec copy test.mp4"

Removing the avi container fixed the error, yet I think it would be valuable to somehow protect the code from such errors (single tracking procedure can take many hours so such behavious can cause problems).