Kinetic branch on Ubuntu 18.04 and using ROS-melodic

[Tariq-Abuhashim]

What parameters did you use to generate the sample Valldemossa result ? I have run this source on the dataset and the result looks blurred.

[emiliofidalgo]

Hi!

Did you check the _blendseams option before starting the mosaicing process? This option should be checked. Without this, you will obtain just a draft of the mosaic to see if it is topologically coherent.

[Tariq-Abuhashim]

Hi Emilio.

I don't use rqt.

I launch the node: roslaunch bimos bimos.launch Which has blend_seams set to "true" as a default.

Then, I start the mosaic by calling: rostopic pub /bimos_node/init_mosaic std_msgs/Empty I stop that message as soon as text shows up in terminal.

Then, after frames processing is done, I start optimisation and blending by calling: rostopic pub /bimos_node/optim_and_blend std_msgs/Empty

So, I assume that the blend_seams option is already checked.

I am using ROS melodic on Ubuntu 18.04 I have also used rqt, but the result was also blurred. I have noticed that checking or unchecking the blend_seam option doesn't have any noticeable effect on the result.

Here is a sample mosaic https://ibb.co/xL50t8G

[emiliofidalgo]

Hi Tariq,

I would say that is a problema related to the seams option, but it is weird that it doesn't work for you. I checked the code and I found the following line:

https://github.com/emiliofidalgo/bimos/blob/e03ef247fd220de7c7a90df81b574c5ff860c798/src/node/bimos_node.cpp#L260

As can be seen, the line where the parameters should be read is commented, and, by default, the instance of Params sets blend_seam to false. Could you please check the actual value of the blend_seams option after running bimos.launch?:

rosparam get /

[Tariq-Abuhashim]

Sorry, I forgot to mention that I am looking at the kinetic branch. bimos_node doesn't have that line on that branch.

output of rosparam get / bimos_node: {batch: true, batch_images_dir: /datasets/ODEMAR/130f/, blend_exp: false, blend_seams: true, img_descriptor: ORB_ORB, kf_min_inliers: 100, kf_overlap: 0.2, lc_delay_kfs: 7, match_ratio: 0.8, max_reproj_error: 30.0, min_inliers: 35, nkeypoints: 3000, optim_every_kfs: 30, pub_debug_info: true, working_dir: /home/emilio/Escritorio/Serialization/Mosaicing/} rosdistro: 'kinetic

I reinstalled Ubuntu 16.04 and ros kinetic to verify if its melodic issue. Does the master branch go with ros melodic? because it doesn't compile against kinetic for sure.

[Tariq-Abuhashim]

I think its OpenCV GPU issue. I probably don't have OpenCV GPU support compiled. I will verify.

    //#if defined(HAVE_OPENCV_GPU)
    cv::Ptr<cv::detail::ExposureCompensator> compensator;
    if (p->blend_exp)
    {
        ROS_INFO("[blender] Compensating exposures ...");
        for (size_t i = 0; images_warped.size(); ++i)
        {
            images_warped_umat[i] = images_warped[i].getUMat( cv::ACCESS_READ );
            masks_warped_umat[i] = masks_warped[i].getUMat( cv::ACCESS_READ );
        }
        compensator = cv::detail::ExposureCompensator::createDefault(cv::detail::ExposureCompensator::GAIN_BLOCKS);
        compensator->feed(corners, images_warped_umat, masks_warped_umat);
    }

    //if (p->blend_seams)
    //{
        ROS_INFO("[blender] Finding seams ...");
        cv::Ptr<cv::detail::SeamFinder> seam_finder = new cv::detail::GraphCutSeamFinder(cv::detail::GraphCutSeamFinderBase::COST_COLOR);
        seam_finder->find(images_warped_f, corners, masks_warped_umat);
        images_warped_f.clear();
    //}
     //#endif

On the other hand, for the kinetic branch, CMakeLists.txt is missing OpenMP include/link lines:

    if(OPENMP_FOUND)
        set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${OpenMP_C_FLAGS}")
        set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${OpenMP_CXX_FLAGS}")
        set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${OpenMP_EXE_LINKER_FLAGS}")
    endif()

[Tariq-Abuhashim]

Hi Emilio; I think that I have resolved this issue. There is a bug or error in functionality in the kinetic branch. From your description, blend_seams and blend_exp should be independent. However, in the kinetic branch, blend.cpp has: if (p->blend_seams) then seam_finder->find(images_warped_f, corners, masks_warped_umat);

images_warped_f is of type cv::UMat already and is initialized correctly: std::vector\<cv::UMat> images_warped_f(nimages); and filled in the "warping images" loop images_warped_f[i] = tmp_img.getUMat( cv::ACCESS_READ );

however, masks_warped_umat is initialized as std::vector\<cv::UMat> images_warped_umat(nimages); but its only getting filled if (p->blend_exp) is true.

hence, blend_exp and blend_seam should both be enabled OR masks_warped_umat[i] = masks_warped[i].getUMat( cv::ACCESS_READ ); should be moved to the "warping images" loop instead. Otherwise images_warped_umat will be empty if passed to seam_finder->find().

Finally, HAVE_OPENCV_GPU flag isn't defined, but that's an easy fix. HAVE_OPENCV_CUDAWARPING instead of HAVE_OPENCV_GPU for opencv-3.4

Thanks for this great piece of work.

[emiliofidalgo]

Hi Tariq,

Sorry for my late reply. Thank you for finding the error. The kinetic branch was a pull request and perhaps it includes some bugs that I didn't check enough:

https://github.com/emiliofidalgo/bimos/pull/2

If you can to pull-request the changes to the kinetic branch, I'll be more than happy to accept them. Otherwise, I hope you can use the code and you find it useful.

[Tariq-Abuhashim]

Yeh, I am happy to do that. Thanks.

see #11