CPU CascadeClassifier with HOG not working anymore with opencv 3.0.0.rc1

[opencv-pushbot]

Transferred from http://code.opencv.org/issues/4336

|| Valentino Proietti on 2015-05-14 09:47
|| Priority: Normal
|| Affected: branch 'master' (3.0-dev)
|| Category: objdetect
|| Tracker: Bug
|| Difficulty: Hard
|| PR: 
|| Platform: x64 / Linux

CPU CascadeClassifier with HOG not working anymore with opencv 3.0.0.rc1

Hi,
While moving from opencv 2.4.10 to opencv 3.0.0rc1 I found that the CPU CascadeClassifier is not working anymore with HOG.
Digging into the code I discovered that the HOGEvaluator is completely missing in 3.0.

This is the FeatureEvaluator::create() method from the *2.4* version:
<pre>
Ptr<FeatureEvaluator> FeatureEvaluator::create( int featureType )
{
    return featureType == HAAR ? Ptr<FeatureEvaluator>(new HaarEvaluator) :
        featureType == LBP ? Ptr<FeatureEvaluator>(new LBPEvaluator) :
        featureType == HOG ? Ptr<FeatureEvaluator>(new HOGEvaluator) :
        Ptr<FeatureEvaluator>();
}
</pre>

And this is the same method from the *3.0* version:
<pre>
Ptr<FeatureEvaluator> FeatureEvaluator::create( int featureType )
{
    return featureType == HAAR ? Ptr<FeatureEvaluator>(new HaarEvaluator) :
        featureType == LBP ? Ptr<FeatureEvaluator>(new LBPEvaluator) :
        Ptr<FeatureEvaluator>();
}
</pre>

They are both extracted from the "cascadedetect.cpp" source file of the relative version.

The "opencv_traincascade" utility included in the 3.0.0rc1 version still permits us to use HOG for training but we don't know how to use the training results anymore. 
Is it a bug or a decision ?
If the latter: how are supposed to be used files produced by "opencv_traincascade" with HOG ?

Thank you
Valentino

History

Alex D on 2015-05-15 12:58

Valentino Proietti wrote:
> Hi,
> While moving from opencv 2.4.10 to opencv 3.0.0rc1 I found that the CPU CascadeClassifier is not working anymore with HOG.
> Digging into the code I discovered that the HOGEvaluator is completely missing in 3.0.
> 
> This is the FeatureEvaluator::create() method from the *2.4* version:
> [...]
> 
> And this is the same method from the *3.0* version:
> [...]
> 
> They are both extracted from the "cascadedetect.cpp" source file of the relative version.
> 
> The "opencv_traincascade" utility included in the 3.0.0rc1 version still permits us to use HOG for training but we don't know how to use the training results anymore. 
> Is it a bug or a decision ?
> If the latter: how are supposed to be used files produced by "opencv_traincascade" with HOG ?
> 
> Thank you
> Valentino

I have exactly the same problem.

Vadim Pisarevsky on 2015-05-18 14:30

-   Category set to objdetect

Vadim Pisarevsky on 2015-05-28 14:24

We decided to drop the current HOG cascades in OpenCV 3.x. The implemented HOG features are quite weird - different from Dalal's interpretation of HOG, different from P. Dollar integral channel features. In xobjdetect we slowly grow superior ICF/ACF+WaldBoost-based detector, which is there already and will be improved during 2015.
-   Difficulty set to Hard
-   Assignee set to Vadim Pisarevsky
-   Status changed from New to Cancelled

Valentino Proietti on 2015-05-29 09:56

I'm just wondering why this "design decision" has not been evidenced as needed on the release notes, but may be it's just my fault.

Thank you very much indeed Vadim !!!

Steven Puttemans on 2015-06-10 09:36

Valentino Proietti wrote:
> I'm just wondering why this "design decision" has not been evidenced as needed on the release notes, but may be it's just my fault.
> 
> Thank you very much indeed Vadim !!!

Actually that is a very good point. It should be mentioned if functionality is dropped. We should remove the ability to train HOG features than also from the training interface. Will take a look at it later on! I will reopen this bugreport simply so I can link the PR for the deletion of the functionality.
-   Assignee changed from Vadim Pisarevsky to Steven Puttemans
-   Status changed from Cancelled to Open

[qinjian623]

Hi, I'm wondering whether there are any plans to fix this issue?

[Dikay900]

The corresponding code was completely rewritten in 3.0, while maintaining LBP and HAAR as Vadim already stated: We decided to drop the current HOG cascades in OpenCV 3.x. The implemented HOG features are quite weird - different from Dalal's interpretation of HOG, different from P. Dollar integral channel features. In xobjdetect we slowly grow superior ICF/ACF+WaldBoost-based detector, which is there already and will be improved during 2015.

So I would suggest that you either use the 2.4 version of Opencv and use HOG there if needed, else i would strongly recommend evaluating other mentioned algorithms form the opencv_contrib repo.

With this said i would recommend closing this issue with "wont fix" since it doesn't seem to be productive to implement a weird form of an algorithm to the codebase.

[Vale68]

Looks like the "ICF/ACF+WaldBoost-based detector" is disappeared in opencv 3.1, at least for what concerns the ICF and ACF features, there is still a WaldBoost-based detector but it is using LBP (I guess). Anyway it is not comparable in functionalities to the HOG+CascadeClassifier combination, neither for training nor for object detection. In my opinion I think it has been a very bad decision to drop the HOG support in CascadeClassifier since there is not a real substitution.

[Dikay900]

So using the 2.4 branch is also not an option for you? This would be the only chance for using the HOG features i guess.

[qinjian623]

Here is a HOG feature implement on the master branch of opencv, https://github.com/Itseez/opencv/blob/master/modules/objdetect/src/hog.cpp I don't know whether this means the HOG feature will be added in the future.

And the reason of dropping the code is The implemented HOG features are quite weird , so if anyone could implement the code in a not-weird way, will HOG be added into the codebase. Or is any PR of HOG feature implement still welcome?

[Vale68]

Yes the 2.4 branch is ok for me, I'll stay on it. Thank you

[Dikay900]

I just looked through the history of the file and it seems like the statement of vadim was written while he worked through this code i guess. Well i cannot answer if a new HOG implementation will be accepted but in the past new implementations will always be accepted if they are correct. Maybe this code will be only accepted into the opencv_contrib repo but this doesn't really matter. Easiest way for now would be using the 2.4 branch if you just want to use it else feel free to correct the hog implementation or start a new one.

[hfarazi]

It seems that HOGCascade implementation in opencv2.4 is similar to this approach: "Fast Human Detection Using a Cascade of Histograms of Oriented Gradients" http://lear.inrialpes.fr/people/triggs/student/vj/zhu-cvpr06.pdf

So, it seems that in contrast to what @Dikay900 and @vpisarev mentioned, it is a valid implementation.

[elchan]

Removing a useful feature like HOG Cascade is not a good approach. Instead if the current developers don't understand the original code, and the original developers are not available to help, then we should just leave the feature in, until we can find someone who can understand and take over the code. The SVM based HOGDescriptor is great, but as researchers, we want to train our own HOGDescriptors and not just make use of the canned vectors. Is it not possible to just add back the version 2.4 Cascade based HOG classifier code?

[StevenPuttemans]

@elchan in my opinion you are mixing 2 things up. HOG based cascade classifiers are not the same as SVM based HOGDescriptors. It is perfectly possible to train your own HOGDescriptors combined with SVM if you want, by following this tutorial.

As to the cascade classification. As long as no-one provides a PR with the combination of the current HOG feature descriptor and the cascade classifier pipeline, I am afraid that it will not get back in. However, if someone starts tackling this, I am up for help, because I would love to generalize the interface to allow even more features to be added to cascade classifier training.

[elchan]

@StevenPuttemans Thanks that link. It will be helpful, if and when I implement a SVM HOG training program. Of course, I know that SVM HOG is different from Cascade HOG. I'm just saying that we did have the CascadeClassifier with HOG runing fine in 2.4. So why take out the functionality just because no one is there to maintain it or someone thinks the implementation was not what we had expected. I'm all for adding it back, and after my current work assignment may take on the task of adding it back.

[StevenPuttemans]

@vpisarev I guess the only way to get this discussion anywhere is to get your opinion on why you actually removed the HOG cascade as is. If not we might just better reintroduce it or fix it where needed.

have the CascadeClassifier with HOG runing fine in 2.4

Actually it is not because it outputs results that the results are correct :)

So why take out the functionality just because no one is there to maintain it or someone thinks the implementation was not what we had expected

I am pretty sure it was not taken out because of lack of support since tons of modules have a lack of support besides from a core dev/community team. As to thinking it was not as expected, the core developpers all work for pretty decent and top level vision companies, who do not remove something just because they think it is faulty. They will have a valid reason for that.

and after my current work assignment may take on the task of adding it back

Do spare yourself the work and wait until @vpisarev gets back on this. Else your efforts might just be for nothing. Ask him how OpenCV dev team sees a working implementation and get your ideas in a same direction first. That will get us furthest as possible :)

[cmrigney]

Any progress on this?

[StevenPuttemans]

Any progress on this?

I guess not since @vpisarev did not reply yet. Being patient is hard, but core devs are quite occupied xD

[elchan]

Well just another point I want to make is that as an alternative, to HOG, I'd been using HAAR CascadeClassifier in my project. And the results are not nearly as good for people detection. We definitely need to resurrect the HOG detector within OpenCV.

[cmrigney]

People detection is the application where I need it also. Which HOG is great for, but HAAR not so much. The existing HOG cascade worked fairly well and was fast.

[StevenPuttemans]

For people detection there is currently a DPM detector available. And BTW, HOG+SVM is still available, which is way better than HOG cascades...

[cmrigney]

@StevenPuttemans The only problem with HOG+SVM is it's not real-time performance. HOG cascade was real time. Unless there is a way for HOG+SVM to reject subwindows early... but then I would think that would make it a cascade. I'll check out the DPM too, thanks.

[hfarazi]

DPM detector is famous for its accuracy. It is much slower than HOG+Cascade. DPM cascade is a little bit faster than DPM but still not good enough.

[StevenPuttemans]

@cmrigney HOG+SVM is not real time? How are you evaluating your case? Because on any midsize resolution, with some sort of mask defining and release build, it runs just real time here.

@hfarazi well DPM in OpenCV is also far from optimized. One of the reasons why there are so many other implementations out there.

[hfarazi]

It is obvious that each method has its own pros and cons. For many of us, HOG+Cascade was just fine. It was definitely way faster than HOG+SVM. I still haven't heard a convincing argument why we removed a working, valid and optimized implementation.

[cmrigney]

@StevenPuttemans I'm running on an ARM embedded device and I tested about 1 year ago. HOG cascade gave me about 15-20 FPS and HOG+SVM was giving me around 1-3 FPS. The resolution is 320x240.

[StevenPuttemans]

I still haven't heard a convincing argument why we removed a working, valid and optimized implementation.

Agreed, however since I am not a core dev, I cannot supply you with the answer ^_^

@cmrigney on embedded platforms the FFLD person detector combined with a Kalman tracker can solve some of your speed issues maybe.

[cmrigney]

@StevenPuttemans Thanks for the advice! I'll dig into it.

Hopefully we can bring the HOG cascade back.

[Schmetzler]

Hey guys, I also needed the HOG cascade for my project, so I "ported/copied" it from the OpenCV2.x branch. I created a repository for it: https://github.com/Schmetzler/opencv3_CascadeHOG It is not as integrated as it was in OpenCV2, but at least its usable.

[cmrigney]

@Schmetzler Thanks!

I'm betting we could do the same thing to put the HOG code back into OpenCV3. Is it acceptable to forward port the OpenCV2 code core devs? I may be able to take up this task if so.

[amlandas78]

What is the weak classifier in this (https://github.com/Schmetzler/opencv3_CascadeHOG) cascade classifier implementation?

[cmrigney]

@amlandas78 I'm pretty sure the weak classifier is a stump using a HOG feature. I don't know the details of the implementation though.

[StevenPuttemans]

@amlandas78 basically each stage element is a weak cascade. You can follow this parameter throughout the code. It all depends on how the model was trained by the author, to see if it are stumps or not ...

[amlandas78]

Can I use SVM as weak classifier in this case? If yes, what are the changes required? If I use stump as weak classifier what should be the count? Should it be equal to the number of features I am extracting from a single training sample?

[StevenPuttemans]

Euhm you are mixing stuff up, if you need HOG + SVM, then you will need something completely different. The cascade classifier interface is based on boosting ...

[amlandas78]

But in www.merl.com/reports/docs/TR2006-068.pdf the weak classifiers are separating hyperplane computed using linear SVM.

[StevenPuttemans]

Correct, but the cascades in OpenCV are based on the Viola and Jones paper. They just added HOG as another feature representation. We are talking about two different things here in my opinion.

[StevenPuttemans]

Ow wait, let me correct that, they do use AdaBoost for feature selection, but SVM instead of decision trees. I guess for that you will need to break open the code of OpenCV if you want that.

[amlandas78]

Exactly. Thanks for your valuable reply and time.

[cmrigney]

@amlandas78 Please let me know if you manage to implement that, as I was looking to do something similar.

[amlandas78]

Ya sure @cmrigney .

[anonym24]

I hate the situation, get it back already!

[StevenPuttemans]

@anonym24 If you so desperately need it, why not provide the PR yourself?