The code "room_boost_->predict" crashes.

[sxy370921]

When I set parameter room_segmentation_algorithm to 4 or 5 to run “room_segmentation_action_client" in ROS Kinetic, the program crashed and the server was restarted. The problems are described in https://github.com/ipa320/ipa_coverage_planning/issues/12#issue-1036255549. When I looked for the cause of the problems, I found that all programs crashed with "roomboost->predict". The version of my Opencv is 3.3.1. So, what happened to AdaBoost in OpenCV?

[ipa-rmb]

Alright, it seems we had this problem already in the past, but by that time we had solved it. @ipa-foj , do you remember how? Did we deliver any pre-trained models that need to be deleted because they were trained with the other OpenCV version? I could imagine something like that. @sxy370921 what you could try is to run a new training cycle for the AdaBoost-based approaches to create new compatible models.

[ipa-rmb]

I just checked, we deliver pre-trained AdaBoost-models. I could well-imagine, those were trainined in an older OpenCV-version and after a design update in OpenCV, now they do not work anymore. Please try to train those approaches from scratch. Or do you have any better bet, @ipa-foj ?

[sxy370921]

Interestingly, the training function has the same problem. However, the crash in the training function has been resolved. This is due to uninitialized pointers in the new opencv version. The solutions are as follows:

    cv::Ptr<cv::ml::Boost> model;
    std::cout<<"test001"<<std::endl;
    model = cv::ml::Boost::create();
    model->setBoostType(cv::ml::Boost::GENTLE);
    model->setWeakCount(350);
    model->setWeightTrimRate(0);
    model->setMaxDepth(2);
    model->setUseSurrogates(false);
    // model->setPriors(Mat(priors));
    std::cout<<"test002"<<std::endl;
    model->train(hallway_features_mat, cv::ml::ROW_SAMPLE, hallway_labels_mat);

However, after I successfully retrained, the prediction process still crashed. The problem "OpenCV Error: Assertion failed (!fixedSize() || ((Mat*)obj)->size.operator()() == Size(_cols, _rows)) in create, file /build/opencv-L2vuMj/opencv-3.2.0+dfsg/modules/core/src/matrix.cpp, line 2287" always occurs. When I changed room_boost_->predict(features_mat, cv::Mat(), cv::ml::Boost::RAW_OUTPUT); into room_boost_->predict(features_mat, cv::noArray(), cv::ml::Boost::RAW_OUTPUT);, the program no longer crashed. But I don't know why all free grids was segmented to one room. @ipa-rmb

[ipa-rmb]

Hi Florian,

das Problem sieht komisch aus. Hast du eventuell eine Idee? Kannst du ihm was antworten oder zumindest sagen, wann du dir das anschauen könntest?

Ich schaffe es diese Woche auf jeden Fall nicht mehr.

Danke und viele Grüße, Richard

Von: sxy370921 @.> Gesendet: Mittwoch, 3. November 2021 02:40 An: ipa320/ipa_coverage_planning @.> Cc: Bormann, Richard @.>; Mention @.> Betreff: Re: [ipa320/ipa_coverage_planning] The code "roomboost->predict" crashes. (Issue #14)

Interestingly, the training function has the same problem. Interestingly, the training function has the same problem. However, the crash in the training function has been resolved. This is due to uninitialized pointers in the new opencv version. The solutions are as follows:

cv::Ptr<cv::ml::Boost> model;

std::cout<<"test001"<<std::endl;

model = cv::ml::Boost::create();

model->setBoostType(cv::ml::Boost::GENTLE);

model->setWeakCount(350);

model->setWeightTrimRate(0);

model->setMaxDepth(2);

model->setUseSurrogates(false);

// model->setPriors(Mat(priors));

    std::cout<<"test002"<<std::endl;

model->train(hallway_features_mat, cv::ml::ROW_SAMPLE, hallway_labels_mat);

However, after I successfully retrained, the prediction process still crashed. The problem "OpenCV Error: Assertion failed (!fixedSize() || ((Mat*)obj)->size.operator()() == Size(_cols, _rows)) in create, file /build/opencv-L2vuMj/opencv-3.2.0+dfsg/modules/core/src/matrix.cpp, line 2287" always occurs. When I changed roomboost->predict(features_mat, cv::Mat(), cv::ml::Boost::RAW_OUTPUT); into roomboost->predict(features_mat, cv::noArray(), cv::ml::Boost::RAW_OUTPUT);, the program no longer crashed. But I don't know why all free grids was segmented to one room. @ipa-rmbhttps://github.com/ipa-rmb [image]https://user-images.githubusercontent.com/46050164/139999060-24205e56-fdb9-4efc-8110-cb37ee05e32e.png

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ipa320/ipa_coverage_planning/issues/14#issuecomment-958590209, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAE55KXYBANIP3ZBKJJEZZLUKCHFFANCNFSM5HFOVQWA. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[ipa-foj]

This issue might really come from changes in the OpenCV version. I'm gonna look into this issue, but I can only invest time from tomorrow on earliest... I will keep you updated.

Regarding the results of retraining: The semantics based segmentation tries to classify all pixel in either room, hallway or doorway. In this case it looks like all the extracted pixel features are similar to what the algorithm learnt for one of these classes, so it cannot distinguish between anything. Maybe one just has to fix the parameters of the training, but in the end the other algorithms will produce more reliable results in a faster way.

[ipa-foj]

After a quick look it seems like there is a missing train function call for the room AdaBoost object is not called and a default model is stored (https://github.com/ipa320/ipa_coverage_planning/blob/melodic_dev/ipa_room_segmentation/common/src/adaboost_classifier.cpp#L187). This would explain why everything is marked as one "class" in the segmentation. Could you try to add the train function as in line 166? Currently we are in the progress to switch to Noetic and there the OpenCV version changes to 4, so it's a little more effort for me to reproduce this issue..

[sxy370921]

There are more problems for predicting function. The program didn't stop crashing until I changed room_boost_->predict(features_mat, cv::Mat(), cv::ml::Boost::RAW_OUTPUT) into room_boost_->predict(features_mat, cv::noArray(), cv::ml::Boost::RAW_OUTPUT). After changing I found that the predict function outputs are 1 and 0 instead of -1 and 1, which leads to all free grids being segmented to one room. I can only forcibly replace the Adaboost output 0 with - 1 to enter the subsequent processing. In addition, my OpenCV version is 3.3.1. After I make the above changes, the output results are shown in the figure below. There is a difference between this result and the result you give on the website. Is the result right and why do the above changes happen？ @ipa-foj

[ipa-foj]

Thanks for testing and letting us know, we will add the changes and test it on our system again. In principle the output seems to be plausible, the changes in the output might be coming from different parameters of training (this method was only ported to Melodic without really thoroughly testing it as the results and speed are not really the best choice for our applications). But I guess it is usable in this form, correct?

[ipa-foj]

I was finally able to look into the issues described here for algorithm 4, turns out some parts of the OpenCV code didn't work as we thought they would in the new version. I added the changes to address them in my fork, branch noetic_dev. We are still checking other possible issues and how to manage versioning in the main repository here, but for now you could use this code if you want to use algorithm 4.