Adds converter task to haar cascade data from OpenCV

[eduardolundgren]

Adds build task that converts new OpenCV training data XML to tracking.js format. That would allow importing all training data available (https://github.com/Itseez/opencv/tree/master/data/haarcascades) to run on the web.

After OpenCV changed the old format to this new XML we need to figure out how this data is placed in order to convert it:

New format: https://github.com/Itseez/opencv/blob/master/data/haarcascades/haarcascade_frontalface_alt2.xml#L55

tracking.js format:

[20,20,0.822689414024353,3,0,2,3,7,14,4...

[zenorocha]

+1

[cirocosta]

:+1: . Seems like the pot of gold lives here: opencv/modules/objdetect/src/cascadedetect_convert.cpp (i added some comments, here is the gist)

It is the source code for the transformation old-->new. From that we need to do almost the inverse (as the array described by @eduardolundgren seems pretty similar to the old representation - and there is already a converter for that)

[cirocosta]

@eduardolundgren, as i understood, the structure that tracking.js uses goes like this:

[
    [parent, stageThreshold, 
        [ rect1[0], rect1[1], rect1[2], rect1[3], rect1[4], 
            rect2[0], rect2[1], rect2[2], rect2[3], rect2[4],
            threshold, left_val, right_val]],
    ...
]

is this parent being used for something? The reason for asking this is the fact that i think that parent is not used anymore at the new version.

We can see at the cascadedetect_convert.cpp this preprocessor statement: #define ICV_HAAR_PARENT_NAME "parent". Although, searching for ICV_HAAR_PARENT_NAME returns nothing.

[cirocosta]

I may have misunderstood something :cry: One more thing: the desired format is that one shown in assets/opencv_haarcascade_converter.html or that one being used by ViolaJones.js ? They are not the same, right?

[eduardolundgren]

I have this file here which converts to the new tracking.js format from the old xml: https://gist.github.com/eduardolundgren/cdf61a41c5fe114ca558

Ignore the format, it's a temporary hacky convertor.

[eduardolundgren]

@cirocosta Forget about the previous gist I pasted, it's a very old version and does not represent what we have, unfortunately we have lost the temporary converter, we need to reverse engineer it.

[cirocosta]

hmmm, that's fine @eduardolundgren. My question is more about the result array that tracking.js needs. I didn't get what are the correct params of [20,20,0.822689414024353,3,0,2,3,7,14,4 .. ]

[eduardolundgren]

The first two values [20,20 are the initial width and height values for the evaluation block.

Then the following is stageThreshold, then the nodeLength (numbers of rectangles in this stage), see https://github.com/eduardolundgren/tracking.js/blob/master/src/detection/ViolaJones.js#L153-L154.

In the method evalStage_ the incrementation of w++ walks through the training data array, this method gives all the hints needed to map with the old format, see https://raw.githubusercontent.com/eduardolundgren/tracking.js/master/assets/opencv_haarcascade_frontalface_alt.js (note this is a 1:1 conversion from the old OpenCV XML to JSON format).

The new openCV XML format has a different structure, and we need to figure this out in order to map to the linear array like https://github.com/eduardolundgren/tracking.js/blob/master/src/detection/training/haar/face.js.

Note that the tracking.js format is not using anymore the nested array structure:

[
    [parent, stageThreshold, 
        [ rect1[0], rect1[1], rect1[2], rect1[3], rect1[4], 
            rect2[0], rect2[1], rect2[2], rect2[3], rect2[4],
            threshold, left_val, right_val]],
    ...
]

Therefore, this conversor https://github.com/eduardolundgren/tracking.js/blob/master/assets/opencv_haarcascade_converter.html is no longer valid and will be deleted once we have a conversor task.

[cirocosta]

Thanks @eduardolundgren ! I've created a conversor (https://github.com/cirocosta/cvc-converter), it's already using transform streams (although not very efficiently) so that we can easily attach to the build process. Later i'll try to gather some classifiers and test with tracking js to see if it is working as intended (i.e., recognizing as it should).

What do you think? any tips?

[eduardolundgren]

LGTM. After your tests with tracking.js we could integrate into our build, maybe having a folder containing all the XMLs, then when run build-data we converts from there. What do you think?

Good job :+1:

[cirocosta]

Hello again!

Some progress on this: changed the name of the repo (now gulp-converter-tjs), tests went fine (yeah, it is detecting well with new type of opencv hcc!), maybe tomorrow at night i have something w/ tests going on (current branch: https://github.com/cirocosta/tracking.js/tree/converter-task).

[shabanraza]

hello @eduardolundgren I am using tracking.js in my projects VirtualTryOn it detect eyes on upload image easly but when we take picture from webcam it detect only one eye .pls help me and one more thing i want to use haar xml classifiers to js file..