search

MRPT / GSoC2018-discussions

2 stars 3 forks source link

GUI App for Automatic Extrinsic Calibration of Range and Visual Sensors (Karnik Ram R) #2

Open EduFdez opened 6 years ago

EduFdez commented 6 years ago

Initial description

Range and visual sensors are being increasingly used alongside one another in mobile robots. With their increasing use, calibration techniques that can accurately estimate the 6-DoF transformation between them are becoming increasingly important. In this regard, an end-to-end application with an easy-to-use graphical user interface for the extrinsic calibration of different types of sensors is proposed. The app will be able to calibrate the extrinsics of 3D LiDARs, RGB-D cameras, RGB cameras, and any combination between them. Automatic and target-less calibration algorithms based on line matching, plane matching, and trajectory matching will be implemented and integrated into the app. The user will be able to directly visualize the calibration results inside the app and also compare different algorithms wherever possible, and significantly reduce calibration efforts.

See the GSoC project proposal

Progress: All comments below reflect the interactions between the GSoC student and the mentors.

jlblancoc commented 6 years ago

No problem!

I'm building it now, and I could see the error. It doesn't seem a normal build error, but a parser error or alike: somehow it's related to the .ui files??

jlblancoc commented 6 years ago

if you run it with VERBOSE=1 make -j1 you can see the actual commands being invoked. The error happens in:

cd /home/jlblanco/code/autocalib-sensor-extrinsics/build/gui && /usr/lib/x86_64-linux-gnu/qt5/bin/moc @/home/jlblanco/code/autocalib-sensor-extrinsics/build/gui/moc_CMainWindow.cpp_parameters
home/jlblanco/code/mrpt/libs/serialization/include/mrpt/serialization/serialization_frwds.:11: Parse error at "mrpt"
gui/CMakeFiles/autocalib_sensor_extrinsics.dir/build.make:61: recipe for target 'gui/moc_CMainWindow.cpp' failed
jlblancoc commented 6 years ago

The error must come from #include <observation_tree/CObservationTreeModel.h> in CMainWindow.h.

Start commenting out blocks of #includes and definitions until the error goes, so you can find it...

jlblancoc commented 6 years ago

Well, you were right: the problem is the use of the nested namespaces namespace mrpt::foo in headers (cc: @bergercookie , @jolting ).

But it's not an error at the gcc level,but at the Qt moc level (!). It's probably worth investigating if a newer version of qt solved it, and if not, please, feel free of opening a bug to Qt.

Anyway, to make progress: try to refactor things such that no .ui files (and their associated .h) needs to include MRPT headers. A simple PIMPL pointer to an "implementation" class would suffice, with that implementation class having a normal .h & .cpp without limitations.

Congrats, you found a Qt bug! ;-)

karnikram commented 6 years ago

Thanks for the help @jlblancoc!

It seems like support for nested namespaces was added in Qt 5.9 if I'm not wrong - https://codereview.qt-project.org/#/c/187819/ Should I upgrade our project to 5.9?

For now as a work around I'm using the preprocessor symbol Q_MOC_RUN to make the moc skip all the non-qt header files.

I've pushed these changes to my fork just to have a working build.

EduFdez commented 6 years ago

Hi @karnikram

Should I upgrade our project to 5.9?

As @jlblancoc said, you can try to upgrade Qt and check if the Qt bug is fixed in the new version.

karnikram commented 6 years ago

Hi,

I've installed Qt 5.9.2 (compatible with nested namespaces) in my home directory and have been trying to link our project to these new libraries but in vain.

I've set Qt5Widgets_DIR to /home/karnik/Tools/Qt/5.9.2/gcc_64/lib/cmake/Qt5Widgets/ as recommended here. Now Qt5Widgets_VERSION shows 5.9.2.

But strangely, Qt5Widgets_INCLUDE_DIRS still points to the old system installed Qt 5.5 libraries that are in /usr/include (included by default in Ubuntu 16.04) 

usr/include/x86_64-linux-gnu/qt5//usr/include/x86_64-linux-gnu/qt5/QtWidgets/usr/include/x86_64-linux-gnu/qt5/QtGui/usr/include/x86_64-linux-gnu/qt5/QtCore/usr/lib/x86_64-linux-gnu/qt5//mkspecs/linux-g++-64/usr/include

and our project still fails to compile because of the same nested namespace parsing error.

Any idea how to get cmake to point to the correct Qt version?

I also set CMAKE_PREFIX_PATH to/home/karnik/Tools/Qt/5.9.2/gcc_lib/cmake/ but there's no change.

Update: I've posted the issue on SO

EduFdez commented 6 years ago

I guess that you need to delete the cache in cmake to update correctly to the new version of Qt. Since this does not work for you, there is the option to write your own FindQt.cmake file to include the correct paths. But I advice you to avoid wasting time with this issue of the Qt version, you may point back to the previous Qt5.5 and concentrate on the functionalities that you wrote in the timeline of the proposal.

karnikram commented 6 years ago

Hi,

Just a side question.

Right now I'm passing a function object pointing to CViewerContainer::updateText, to CPlaneMatching so that it can display the progress.

But what if updateText was templated, and the specific type is known only within CPlaneMatching. Since we can only have pointers to concrete functions, is there any way to do this apart from passing the CViewerContainer ui object itself?

I'm asking this because I'm now passing a function object pointing to updateViewer, which could well be templated to handle different pcl point types like pcl::PointXYZRGB, pcl::Normal (not a must)

Hope I made sense.

EduFdez commented 6 years ago

Hi @karnikram It is possible to template that function, you just need to specify the full signature of the function for calling it and for passing it as a parameter. But your dependency structure of the GUI does not seem adequate, I tried to guide you about this in my previous comment about the cyclic inclusion but I was not clear enough. I'll try to improve my explanation here: CPlaneMatching should not have a pointer to CViewerContainer::updateText. Instead, CViewerContainer should be able to access the relevant content of CPlaneMatching. You can also use a null pointer to CPlaneMatching to indicate that CPlaneMatching is not initialized. Another way to see this is by designing the calibration app with and without GUI, which is also interesting in order to write unit tests for different functionalities like CPlaneMatching, which should not require the GUI. In summary, all the modules (or functionalities) of the calibration app should work correctly outside of the GUI, while the GUI must be able to call them and show the relevant information to the user. Does it make sense now?

karnikram commented 6 years ago

Thanks @EduFdez

CViewerContainer should be able to access the relevant content of CPlaneMatching

But how exactly? You'd suggested using an interface class earlier but I did not fully get that.

EduFdez commented 6 years ago

The idea is to make CPlaneMatching independent from the GUI, but after checking again your code I see that it is already independent. I got confused about it when I saw that you call your updateFunction in the constructor of CPlaneMatching. I'm sorry for the confusion.

My suggestion about the interface class would look like this: CPlaneMatching: does not contain any sendTextUpdate, just the parameters and methods required for plane matching CInterface: has sendTextUpdate and can access the relevant info from CPlaneMatching, CLineMatching, etc., for sending relevant information to the GUI In this way you would isolate better the functionality from the GUI, what would improve readability.

karnikram commented 6 years ago

I got confused about it when I saw that you call your updateFunction in the constructor of CPlaneMatching.

That was also bad naming on my part :)

CInterface: has sendTextUpdate and can access the relevant info from CPlaneMatching, CLineMatching, etc., for sending relevant information to the GUI

Getting clearer. But still a little confused about who should inherit this interface class, and how this can avoid function pointers. Is there any small example that I can look at?

EduFdez commented 6 years ago

I'm not sure if the interface class will help to avoid function pointers as I have almost no experience programming GUIs and I do not know Qt very well. While the inheritance will depend on practical aspects of the application.

The point of the intermediate class is to encapsulate the interface parameters so that the core classes like CPlaneMatching do not need to handle GUI aspects. This should help to structure your code, and to improve readability.

karnikram commented 6 years ago

Hi @EduFdez,

I read more about circular dependencies and behavioral design patterns, and have now completely understood the problem that you were talking about.

I've solved it by implementing a minimal publisher-subscriber pattern where the publisher maintains a list of subscribers and notifies them about any status changes automatically. The subscriber has to implement the CSubscriber interface and register with the publisher. In our case CViewerContainer is a subscriber and CPlaneMatching is the publisher. Now CPlaneMatching is not closely coupled to any class, does not take in any function pointers, and new subscribers can be added easily. There can also be no subscribers and CPlaneMatching can run independently, for example in unit tests.

I've pushed these changes to my fork. Hope this design is better.

Edit: Technically I think this is called the observer pattern and not publisher-subscriber.

karnikram commented 6 years ago

Hi @EduFdez, @jlblancoc

Plane segmentation is now working reasonably well. I had to make some very small changes to a few mrpt lib classes to get it to work. I've submitted these changes to the main mrpt repo as a pull request, hope you can take a look at it.

Meanwhile, I ran into another small issue. I'm able to display the segmented planes properly but for some reason the original point cloud isn't being displayed properly. Its color is not being shown and it is very "rough" even though I'm using the same pcl visualizer object, and even when the planes that are extracted from the very same cloud object are shown properly, and in color.

The display function is here. The original cloud is being displayed from here, and the extracted planes from here. Not sure where the problem is. I'm using PCL 1.7. MAKE_ORGANIZED doesn't make any difference here.

segmented-cloud input-cloud

jlblancoc commented 6 years ago

@EduFdez I leave to you for now taking a look at the project-specific code; I've reviewed the MRPT-specific changes; please @karnikram see the comments there.

EduFdez commented 6 years ago

Thank you @jlblanco, I will start reviewing it today. @karnikram I will also give you some comments soon, and I will have a look at the problem of the point cloud visualization.

karnikram commented 6 years ago

Thanks @EduFdez,

Neglecting the color problem for now, I just pushed some more changes for easier visualization. Clicking on any observation item will now show the corresponding planes overlaid over the original cloud as follows:

viewers

karnikram commented 6 years ago

Minor update: I tested with Qt 5.9.5 (default in Ubuntu 18.04), it doesn't have any issues with nested namespaces. I'm also now developing on Ubuntu 18.04.

karnikram commented 6 years ago

I'm having some problems with PCL 1.8 where this line is causing a segmentation fault arbitrarily (while being de-allocated). 

std::vector<pcl::PlanarRegion<pcl::PointXYZRGBA>, Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA>>> regions;

A gdb backtrace gives,

Thread 1 "autocalib_senso" received signal SIGSEGV, Segmentation fault.
__GI___libc_free (mem=0xb5) at malloc.c:3103
3103    malloc.c: No such file or directory.
(gdb) bt
#0  __GI___libc_free (mem=0xb5) at malloc.c:3103
#1  0x0000555555581d98 in Eigen::internal::handmade_aligned_free (ptr=0x7fffb8012170) at /usr/include/eigen3/Eigen/src/Core/util/Memory.h:98
#2  0x0000555555581df4 in Eigen::internal::aligned_free (ptr=0x7fffb8012170) at /usr/include/eigen3/Eigen/src/Core/util/Memory.h:179
#3  0x00005555555bc370 in Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> >::deallocate (this=0x7fffffffc9b0, p=0x7fffb8012170) at /usr/include/eigen3/Eigen/src/Core/util/Memory.h:747
#4  0x00005555555b9b92 in std::allocator_traits<Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >::deallocate (__a=..., __p=0x7fffb8012170, __n=1) at /usr/include/c++/7/bits/alloc_traits.h:328
#5  0x00005555555b6566 in std::_Vector_base<pcl::PlanarRegion<pcl::PointXYZRGBA>, Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >::_M_deallocate (this=0x7fffffffc9b0, __p=0x7fffb8012170, __n=1)
    at /usr/include/c++/7/bits/stl_vector.h:180
#6  0x00005555555b0726 in std::_Vector_base<pcl::PlanarRegion<pcl::PointXYZRGBA>, Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >::~_Vector_base (this=0x7fffffffc9b0, __in_chrg=<optimized out>)
    at /usr/include/c++/7/bits/stl_vector.h:162
#7  0x00005555555aa055 in std::vector<pcl::PlanarRegion<pcl::PointXYZRGBA>, Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >::~vector (this=0x7fffffffc9b0, __in_chrg=<optimized out>)
    at /usr/include/c++/7/bits/stl_vector.h:435
#8  0x00005555555a5c1c in CPlaneMatching::runSegmentation (this=0x7fffb800fb30, cloud=..., extracted_planes=...) at /home/karnik/code/autocalib-sensor-extrinsics/core/calib_solvers/CPlaneMatching.cpp:131
#9  0x00005555555a5425 in CPlaneMatching::extractPlanes (this=0x7fffb800fb30) at /home/karnik/code/autocalib-sensor-extrinsics/core/calib_solvers/CPlaneMatching.cpp:85
#10 0x0000555555581010 in CMainWindow::runCalib (this=0x7fffffffe0b0, params=...) at /home/karnik/code/autocalib-sensor-extrinsics/gui/CMainWindow.cpp:213
#11 0x00005555555a14ca in CPlaneMatchingConfig::startCalib (this=0x55555738d660) at /home/karnik/code/autocalib-sensor-extrinsics/gui/config/CPlaneMatchingConfig.cpp:35
#12 0x00005555555a4b3d in CPlaneMatchingConfig::qt_static_metacall (_o=0x55555738d660, _c=QMetaObject::InvokeMetaMethod, _id=0, _a=0x7fffffffd3c0)
    at /home/karnik/code/autocalib-sensor-extrinsics/build/gui/config/moc_CPlaneMatchingConfig.cpp:78
#13 0x00007ffff09695b5 in QMetaObject::activate(QObject*, int, int, void**) () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#14 0x00007ffff1040b82 in QAbstractButton::clicked(bool) () from /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#15 0x00007ffff1040d9a in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#16 0x00007ffff104217a in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5
#17 0x00007ffff104236d in QAbstractButton::mouseReleaseEvent(QMouseEvent*) () from /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5

Similar issues have been posted here. I shall try to build PCL from source as suggested and report back.

jlblancoc commented 6 years ago

Try running your app with valgrind to see if it detects any relevant error...

karnikram commented 6 years ago 
==20743== LEAK SUMMARY:
==20743==    definitely lost: 9,349 bytes in 268 blocks
==20743==    indirectly lost: 10,308,868 bytes in 95,106 blocks
==20743==      possibly lost: 5,426,800 bytes in 24 blocks
==20743==    still reachable: 476,914 bytes in 6,151 blocks
==20743==                       of which reachable via heuristic:
==20743==                         newarray           : 4,264 bytes in 1 blocks
==20743==         suppressed: 0 bytes in 0 blocks
==20743== Rerun with --leak-check=full to see details of leaked memory
==20743== 
==20743== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 21 from 1)
==20743== 
==20743== 1 errors in context 1 of 4:
==20743== Invalid read of size 8
==20743==    at 0x135D8D: Eigen::internal::handmade_aligned_free(void*) (Memory.h:98)
==20743==    by 0x135DF3: Eigen::internal::aligned_free(void*) (Memory.h:179)
==20743==    by 0x13D129: void Eigen::internal::conditional_aligned_free<true>(void*) (Memory.h:230)
==20743==    by 0x169898: pcl::search::OrganizedNeighbor<pcl::PointXYZRGBA>::operator delete(void*) (organized.h:276)
==20743==    by 0x1758B3: pcl::search::OrganizedNeighbor<pcl::PointXYZRGBA>::~OrganizedNeighbor() (organized.h:100)
==20743==    by 0x173ABA: void boost::checked_delete<pcl::search::OrganizedNeighbor<pcl::PointXYZRGBA> >(pcl::search::OrganizedNeighbor<pcl::PointXYZRGBA>*) (checked_delete.hpp:34)
==20743==    by 0x180643: boost::detail::sp_counted_impl_p<pcl::search::OrganizedNeighbor<pcl::PointXYZRGBA> >::dispose() (sp_counted_impl.hpp:92)
==20743==    by 0x79C0559: boost::detail::sp_counted_base::release() (in /usr/lib/x86_64-linux-gnu/libpcl_features.so.1.8.1)
==20743==    by 0x7E93A8F: pcl::IntegralImageNormalEstimation<pcl::PointXYZRGBA, pcl::Normal>::~IntegralImageNormalEstimation() (in /usr/lib/x86_64-linux-gnu/libpcl_features.so.1.8.1)
==20743==    by 0x159C48: CPlaneMatching::runSegmentation(boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGBA> > const&, boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGBA> >&) (CPlaneMatching.cpp:107)
==20743==    by 0x159424: CPlaneMatching::extractPlanes() (CPlaneMatching.cpp:85)
==20743==    by 0x13500F: CMainWindow::runCalib(TPlaneMatchingParams) (CMainWindow.cpp:213)
==20743==  Address 0x40375808 is 8 bytes before a block of size 240 alloc'd
==20743==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==20743==    by 0x79C2178: Eigen::internal::aligned_malloc(unsigned long) (in /usr/lib/x86_64-linux-gnu/libpcl_features.so.1.8.1)
==20743==    by 0x7D3E1C9: pcl::Feature<pcl::PointXYZRGBA, pcl::Normal>::initCompute() (in /usr/lib/x86_64-linux-gnu/libpcl_features.so.1.8.1)
==20743==    by 0x15DB3B: pcl::Feature<pcl::PointXYZRGBA, pcl::Normal>::compute(pcl::PointCloud<pcl::Normal>&) (feature.hpp:191)
==20743==    by 0x1597F8: CPlaneMatching::runSegmentation(boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGBA> > const&, boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGBA> >&) (CPlaneMatching.cpp:122)
==20743==    by 0x159424: CPlaneMatching::extractPlanes() (CPlaneMatching.cpp:85)
==20743==    by 0x13500F: CMainWindow::runCalib(TPlaneMatchingParams) (CMainWindow.cpp:213)
==20743==    by 0x1554C9: CPlaneMatchingConfig::startCalib() (CPlaneMatchingConfig.cpp:35)
==20743==    by 0x158B3C: CPlaneMatchingConfig::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (moc_CPlaneMatchingConfig.cpp:78)
==20743==    by 0xC0CF5B4: QMetaObject::activate(QObject*, int, int, void**) (in /usr/lib/x86_64-linux-gnu/libQt5Core.so.5.9.5)
==20743==    by 0xB814B81: QAbstractButton::clicked(bool) (in /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5.9.5)
==20743==    by 0xB814D99: ??? (in /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5.9.5)
==20743== 
==20743== 
==20743== 1 errors in context 2 of 4:
==20743== Invalid read of size 8
==20743==    at 0x135D8D: Eigen::internal::handmade_aligned_free(void*) (Memory.h:98)
==20743==    by 0x135DF3: Eigen::internal::aligned_free(void*) (Memory.h:179)
==20743==    by 0x172681: Eigen::aligned_allocator<pcl::Label>::deallocate(pcl::Label*, unsigned long) (Memory.h:747)
==20743==    by 0x1704E3: std::allocator_traits<Eigen::aligned_allocator<pcl::Label> >::deallocate(Eigen::aligned_allocator<pcl::Label>&, pcl::Label*, unsigned long) (alloc_traits.h:328)
==20743==    by 0x16DD81: std::_Vector_base<pcl::Label, Eigen::aligned_allocator<pcl::Label> >::_M_deallocate(pcl::Label*, unsigned long) (stl_vector.h:180)
==20743==    by 0x16A778: std::_Vector_base<pcl::Label, Eigen::aligned_allocator<pcl::Label> >::~_Vector_base() (stl_vector.h:162)
==20743==    by 0x16499C: std::vector<pcl::Label, Eigen::aligned_allocator<pcl::Label> >::~vector() (stl_vector.h:435)
==20743==    by 0x17054D: pcl::PointCloud<pcl::Label>::~PointCloud() (point_cloud.h:240)
==20743==    by 0x170579: pcl::PointCloud<pcl::Label>::~PointCloud() (point_cloud.h:240)
==20743==    by 0x16DE16: void boost::checked_delete<pcl::PointCloud<pcl::Label> >(pcl::PointCloud<pcl::Label>*) (checked_delete.hpp:34)
==20743==    by 0x180ACB: boost::detail::sp_counted_impl_p<pcl::PointCloud<pcl::Label> >::dispose() (sp_counted_impl.hpp:92)
==20743==    by 0x136346: boost::detail::sp_counted_base::release() (sp_counted_base_std_atomic.hpp:110)
==20743==  Address 0xac5f9038 is 8 bytes before a block of size 1,228,800 alloc'd
==20743==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==20743==    by 0x9A49650: std::vector<pcl::Label, Eigen::aligned_allocator<pcl::Label> >::_M_fill_insert(__gnu_cxx::__normal_iterator<pcl::Label*, std::vector<pcl::Label, Eigen::aligned_allocator<pcl::Label> > >, unsigned long, pcl::Label const&) (in /usr/lib/x86_64-linux-gnu/libpcl_segmentation.so.1.8.1)
==20743==    by 0x9A4AB85: pcl::OrganizedConnectedComponentSegmentation<pcl::PointXYZRGBA, pcl::Label>::segment(pcl::PointCloud<pcl::Label>&, std::vector<pcl::PointIndices, std::allocator<pcl::PointIndices> >&) const (in /usr/lib/x86_64-linux-gnu/libpcl_segmentation.so.1.8.1)
==20743==    by 0x9AB4AAA: pcl::OrganizedMultiPlaneSegmentation<pcl::PointXYZRGBA, pcl::Normal, pcl::Label>::segment(std::vector<pcl::ModelCoefficients, std::allocator<pcl::ModelCoefficients> >&, std::vector<pcl::PointIndices, std::allocator<pcl::PointIndices> >&, std::vector<Eigen::Matrix<float, 4, 1, 0, 4, 1>, Eigen::aligned_allocator<Eigen::Matrix<float, 4, 1, 0, 4, 1> > >&, std::vector<Eigen::Matrix<float, 3, 3, 0, 3, 3>, Eigen::aligned_allocator<Eigen::Matrix<float, 3, 3, 0, 3, 3> > >&, pcl::PointCloud<pcl::Label>&, std::vector<pcl::PointIndices, std::allocator<pcl::PointIndices> >&) (in /usr/lib/x86_64-linux-gnu/libpcl_segmentation.so.1.8.1)
==20743==    by 0x9B39061: pcl::OrganizedMultiPlaneSegmentation<pcl::PointXYZRGBA, pcl::Normal, pcl::Label>::segmentAndRefine(std::vector<pcl::PlanarRegion<pcl::PointXYZRGBA>, Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >&, std::vector<pcl::ModelCoefficients, std::allocator<pcl::ModelCoefficients> >&, std::vector<pcl::PointIndices, std::allocator<pcl::PointIndices> >&, boost::shared_ptr<pcl::PointCloud<pcl::Label> >&, std::vector<pcl::PointIndices, std::allocator<pcl::PointIndices> >&, std::vector<pcl::PointIndices, std::allocator<pcl::PointIndices> >&) (in /usr/lib/x86_64-linux-gnu/libpcl_segmentation.so.1.8.1)
==20743==    by 0x1599A2: CPlaneMatching::runSegmentation(boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGBA> > const&, boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGBA> >&) (CPlaneMatching.cpp:139)
==20743==    by 0x159424: CPlaneMatching::extractPlanes() (CPlaneMatching.cpp:85)
==20743==    by 0x13500F: CMainWindow::runCalib(TPlaneMatchingParams) (CMainWindow.cpp:213)
==20743==    by 0x1554C9: CPlaneMatchingConfig::startCalib() (CPlaneMatchingConfig.cpp:35)
==20743==    by 0x158B3C: CPlaneMatchingConfig::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (moc_CPlaneMatchingConfig.cpp:78)
==20743==    by 0xC0CF5B4: QMetaObject::activate(QObject*, int, int, void**) (in /usr/lib/x86_64-linux-gnu/libQt5Core.so.5.9.5)
==20743==    by 0xB814B81: QAbstractButton::clicked(bool) (in /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5.9.5)
==20743== 
==20743== 
==20743== 1 errors in context 3 of 4:
==20743== Invalid free() / delete / delete[] / realloc()
==20743==    at 0x4C30D3B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==20743==    by 0x135D97: Eigen::internal::handmade_aligned_free(void*) (Memory.h:98)
==20743==    by 0x135DF3: Eigen::internal::aligned_free(void*) (Memory.h:179)
==20743==    by 0x17036F: Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> >::deallocate(pcl::PlanarRegion<pcl::PointXYZRGBA>*, unsigned long) (Memory.h:747)
==20743==    by 0x16DB91: std::allocator_traits<Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >::deallocate(Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> >&, pcl::PlanarRegion<pcl::PointXYZRGBA>*, unsigned long) (alloc_traits.h:328)
==20743==    by 0x16A565: std::_Vector_base<pcl::PlanarRegion<pcl::PointXYZRGBA>, Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >::_M_deallocate(pcl::PlanarRegion<pcl::PointXYZRGBA>*, unsigned long) (stl_vector.h:180)
==20743==    by 0x164725: std::_Vector_base<pcl::PlanarRegion<pcl::PointXYZRGBA>, Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >::~_Vector_base() (stl_vector.h:162)
==20743==    by 0x15E054: std::vector<pcl::PlanarRegion<pcl::PointXYZRGBA>, Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >::~vector() (stl_vector.h:435)
==20743==    by 0x159C1B: CPlaneMatching::runSegmentation(boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGBA> > const&, boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGBA> >&) (CPlaneMatching.cpp:131)
==20743==    by 0x159424: CPlaneMatching::extractPlanes() (CPlaneMatching.cpp:85)
==20743==    by 0x13500F: CMainWindow::runCalib(TPlaneMatchingParams) (CMainWindow.cpp:213)
==20743==    by 0x1554C9: CPlaneMatchingConfig::startCalib() (CPlaneMatchingConfig.cpp:35)
==20743==  Address 0x3b0dcec0 is 32 bytes before a block of size 64 in arena "client"
==20743== 
==20743== 
==20743== 1 errors in context 4 of 4:
==20743== Invalid read of size 8
==20743==    at 0x135D8D: Eigen::internal::handmade_aligned_free(void*) (Memory.h:98)
==20743==    by 0x135DF3: Eigen::internal::aligned_free(void*) (Memory.h:179)
==20743==    by 0x17036F: Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> >::deallocate(pcl::PlanarRegion<pcl::PointXYZRGBA>*, unsigned long) (Memory.h:747)
==20743==    by 0x16DB91: std::allocator_traits<Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >::deallocate(Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> >&, pcl::PlanarRegion<pcl::PointXYZRGBA>*, unsigned long) (alloc_traits.h:328)
==20743==    by 0x16A565: std::_Vector_base<pcl::PlanarRegion<pcl::PointXYZRGBA>, Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >::_M_deallocate(pcl::PlanarRegion<pcl::PointXYZRGBA>*, unsigned long) (stl_vector.h:180)
==20743==    by 0x164725: std::_Vector_base<pcl::PlanarRegion<pcl::PointXYZRGBA>, Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >::~_Vector_base() (stl_vector.h:162)
==20743==    by 0x15E054: std::vector<pcl::PlanarRegion<pcl::PointXYZRGBA>, Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >::~vector() (stl_vector.h:435)
==20743==    by 0x159C1B: CPlaneMatching::runSegmentation(boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGBA> > const&, boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGBA> >&) (CPlaneMatching.cpp:131)
==20743==    by 0x159424: CPlaneMatching::extractPlanes() (CPlaneMatching.cpp:85)
==20743==    by 0x13500F: CMainWindow::runCalib(TPlaneMatchingParams) (CMainWindow.cpp:213)
==20743==    by 0x1554C9: CPlaneMatchingConfig::startCalib() (CPlaneMatchingConfig.cpp:35)
==20743==    by 0x158B3C: CPlaneMatchingConfig::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (moc_CPlaneMatchingConfig.cpp:78)
==20743==  Address 0x3b09f918 is 8 bytes before a block of size 160 alloc'd
==20743==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==20743==    by 0x984FFA8: Eigen::internal::aligned_malloc(unsigned long) (in /usr/lib/x86_64-linux-gnu/libpcl_segmentation.so.1.8.1)
==20743==    by 0x9AF2A68: std::vector<pcl::PlanarRegion<pcl::PointXYZRGBA>, Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >::_M_default_append(unsigned long) (in /usr/lib/x86_64-linux-gnu/libpcl_segmentation.so.1.8.1)
==20743==    by 0x9B390AC: pcl::OrganizedMultiPlaneSegmentation<pcl::PointXYZRGBA, pcl::Normal, pcl::Label>::segmentAndRefine(std::vector<pcl::PlanarRegion<pcl::PointXYZRGBA>, Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >&, std::vector<pcl::ModelCoefficients, std::allocator<pcl::ModelCoefficients> >&, std::vector<pcl::PointIndices, std::allocator<pcl::PointIndices> >&, boost::shared_ptr<pcl::PointCloud<pcl::Label> >&, std::vector<pcl::PointIndices, std::allocator<pcl::PointIndices> >&, std::vector<pcl::PointIndices, std::allocator<pcl::PointIndices> >&) (in /usr/lib/x86_64-linux-gnu/libpcl_segmentation.so.1.8.1)
==20743==    by 0x1599A2: CPlaneMatching::runSegmentation(boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGBA> > const&, boost::shared_ptr<pcl::PointCloud<pcl::PointXYZRGBA> >&) (CPlaneMatching.cpp:139)
==20743==    by 0x159424: CPlaneMatching::extractPlanes() (CPlaneMatching.cpp:85)
==20743==    by 0x13500F: CMainWindow::runCalib(TPlaneMatchingParams) (CMainWindow.cpp:213)
==20743==    by 0x1554C9: CPlaneMatchingConfig::startCalib() (CPlaneMatchingConfig.cpp:35)
==20743==    by 0x158B3C: CPlaneMatchingConfig::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) (moc_CPlaneMatchingConfig.cpp:78)
==20743==    by 0xC0CF5B4: QMetaObject::activate(QObject*, int, int, void**) (in /usr/lib/x86_64-linux-gnu/libQt5Core.so.5.9.5)
==20743==    by 0xB814B81: QAbstractButton::clicked(bool) (in /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5.9.5)
==20743==    by 0xB814D99: ??? (in /usr/lib/x86_64-linux-gnu/libQt5Widgets.so.5.9.5)
==20743== 
--20743-- 
--20743-- used_suppression:     21 X on SUSE11 writev uninit padding /usr/lib/valgrind/default.supp:397
==20743== 
==20743== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 21 from 1)

Trying to make sense of it..

jlblancoc commented 6 years ago

Where's the latest implementation (source code) for CPlaneMatching::runSegmentation so I can take a look at the code and guess what's those invalid free come from?

karnikram commented 6 years ago

@jlblancoc,

It's actually the same implementation that's causing this crash. It was running well on PCL 1.7, but now not on 1.8. Even on 1.8 it ran a couple of times without any error. (!)

This is the line that's causing this arbitrary crash.

EduFdez commented 6 years ago

@karnikram Did you test already if the crash also happens if you link against a compiled version of PCL?

karnikram commented 6 years ago

Not yet @EduFdez, compiling the latest version of PCL is taking forever and it is also exceeding my 4GB physical memory. Even while compiling with just make, in one thread.

EduFdez commented 6 years ago

OK, I will test it and I will tell you if that solves the problem, otherwise I think we should better go back to PCL 1.7 since this kind of error is very difficult to track.

EduFdez commented 6 years ago

I compiled PCL 1.8 from sources, but I am not able to use it in "autocalib-sensor-extrinsics" since CMake complains about Eigen, and I do not know how to solve this properly. The CMake msg is:

The imported target "vtkRenderingPythonTkWidgets" references the file "/usr/lib/x86_64-linux-gnu/libvtkRenderingPythonTkWidgets.so" but this file does not exist. Possible reasons include:

  • The file was deleted, renamed, or moved to another location.
  • An install or uninstall procedure did not complete successfully.
  • The installation package was faulty and contained "/usr/lib/cmake/vtk-6.3/VTKTargets.cmake" but not all the files it references.

The imported target "vtk" references the file "/usr/bin/vtk" but this file does not exist. Possible reasons include:

  • The file was deleted, renamed, or moved to another location.
  • An install or uninstall procedure did not complete successfully.
  • The installation package was faulty and contained "/usr/lib/cmake/vtk-6.3/VTKTargets.cmake" but not all the files it references.

CMake Warning at /home/efernand/libraries/pcl-build/PCLConfig.cmake:143 (find_package): By not providing "FindEigen.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "Eigen", but CMake did not find one.

Could not find a package configuration file provided by "Eigen" with any of the following names:

EigenConfig.cmake
eigen-config.cmake

Add the installation prefix of "Eigen" to CMAKE_PREFIX_PATH or set "Eigen_DIR" to a directory containing one of the above files. If "Eigen" provides a separate development package or SDK, be sure it has been installed. Call Stack (most recent call first): /home/efernand/libraries/pcl-build/PCLConfig.cmake:297 (find_eigen) /home/efernand/libraries/pcl-build/PCLConfig.cmake:507 (find_external_library) CMakeLists.txt:64 (FIND_PACKAGE)

CMake Error at /home/efernand/libraries/pcl-build/PCLConfig.cmake:46 (message): common is required but eigen was not found Call Stack (most recent call first): /home/efernand/libraries/pcl-build/PCLConfig.cmake:346 (pcl_report_not_found) /home/efernand/libraries/pcl-build/PCLConfig.cmake:507 (find_external_library) CMakeLists.txt:64 (FIND_PACKAGE)

Configuring incomplete, errors occurred! See also "/home/efernand/libraries/autocalib-sensor-extrinsics-build/CMakeFiles/CMakeOutput.log". See also "/home/efernand/libraries/autocalib-sensor-extrinsics-build/CMakeFiles/CMakeError.log".

@jlblancoc do you have any idea on how to solve this?

karnikram commented 6 years ago

@EduFdez, are you sure Eigen is installed properly? And where is it placed?

EduFdez commented 6 years ago

@karnikram yes, it is installed but it is not found correctly by PCLConfig.cmake and I'm not sure on how to modify this file. Probably the best thing to do is to come back to the version 1.7 to avoid losing time on this.

EduFdez commented 6 years ago

Hi @karnikram , I fixed some small errors in order to compile the PCL version 1.7.2. (I sent you an email with the patch to fix these errors). I compiled your project against this version of PCL, but unluckily the Segmentation Fault is not solved. You can post your problem on the PCL users mailing list, and I think you should also open an issue in PCL's github.

In the meanwhile, you may comment out the code for the plane segmentation and start working on line segmentation, and also the adaptation of the solver from the code I gave you.

karnikram commented 6 years ago

Hi @EduFdez,

I've already raised an issue on the PCL issues page.

But now since it's not working for you on 1.7.2 either (on 18.04) I think the problem is with the eigen libraries (version 3.3.4-4) in 18.04. Because it worked for me (no segmentation faults) with PCL 1.7.2 on 16.04 that has Eigen version 3.3~beta.

Some people had a similar issue like here and they solved it by using an older version of Eigen. I wanted to try this but I do not know how to make PCL 1.8 binaries to use an older version of Eigen without compiling pcl by myself (which is not possible on my system due to memory constraints).

Let me know what you think.

jlblancoc commented 6 years ago

You both can try just downloading a specific version of Eigen in some directory (NO need to "make install" them, they're just headers), then setting EIGEN3_DIR to the corresponding directory in CMake of the project you want to use a specific version of Eigen.

I've done it in the past without problems.

I think there might not be problems in using the system-provided PCL with a custom version of Eigen.

EduFdez commented 6 years ago

Also, the Segmentation Fault could be caused by using package binaries that have been compiled with different c++ standards, as explained here or here.

karnikram commented 6 years ago

Thanks @jlblancoc, I downloaded eigen-3.2.10 and created an EIGEN3_DIR cmake variable pointing to the headers. But the crash still happens. Is there any way to check which version of Eigen is being used by the pcl binaries?

jlblancoc commented 6 years ago

Is there any way to check which version of Eigen is being used by the pcl binaries?

Yes: going to the ubuntu package website and see its dependencies for your distro. The actual original info is under the file "debian/control" of the pcl package, which you can also download via: "dget http://xxxx/pcl-xxxx.dsc"

Also, the Segmentation Fault could be caused by using package binaries that have been compiled with different c++ standards, as explained here or here.

That also makes sense... If you try building your test case (the one reported to PCL's repo) without the set(CMAKE_CXX_STANDARD 17), with different versions of C++, let's see if the crash happens.

Then it may be caused by a different ABI in the gnu c++ lib. If that's the problem, we'll find a solution, but please first confirm if that's the problem...

EduFdez commented 6 years ago

I tested it by compiling PCL1.8 (master branch) with the -std=c++11 flag and it worked out. I just added the following line to PCL's CMakelists.txt: SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11") I guess it also must work with: set(CMAKE_CXX_STANDARD 17).

@karnikram you will need then to compile it like this, because the option of coming back to ubuntu16 seems a much worse choice. What memory constraints do you have?

karnikram commented 6 years ago

I installed Eigen 3.2.10 headers in /usr/local/include and my example segmentation program listed here works now (with PCL 1.8 binaries). The program is able to use the 3.2.10 headers.

But our project is still crashing. Even though cmake reports using 3.2.10,

-- Checking for module 'eigen3'
--   Found eigen3, version 3.2.10
-- Found eigen: /usr/local/include/eigen3

gdb backtrace suggests it's still using the headers in /usr/include/eigen3 (version 3.3.4-4)

Thread 1 "autocalib_senso" received signal SIGSEGV, Segmentation fault.
__GI___libc_free (mem=0xb1) at malloc.c:3103
3103    malloc.c: No such file or directory.
(gdb) bt
#0  __GI___libc_free (mem=0xb1) at malloc.c:3103
#1  0x0000555555581d58 in Eigen::internal::handmade_aligned_free (ptr=0x555556be54f0)
    at /usr/include/eigen3/Eigen/src/Core/util/Memory.h:98
#2  0x0000555555581db4 in Eigen::internal::aligned_free (ptr=0x555556be54f0)
    at /usr/include/eigen3/Eigen/src/Core/util/Memory.h:179
#3  0x00005555555bc31a in Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> >::deallocate (
    this=0x7fffffffcf10, p=0x555556be54f0) at /usr/include/eigen3/Eigen/src/Core/util/Memory.h:747
#4  0x00005555555b9b3c in std::allocator_traits<Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >::deallocate (__a=..., __p=0x555556be54f0, __n=1) at /usr/include/c++/7/bits/alloc_traits.h:328
#5  0x00005555555b6510 in std::_Vector_base<pcl::PlanarRegion<pcl::PointXYZRGBA>, Eigen::aligned_allocator<pcl::PlanarRegion<pcl::PointXYZRGBA> > >::_M_deallocate (this=0x7fffffffcf10, __p=0x555556be54f0, __n=1)
    at /usr/include/c++/7/bits/stl_vector.h:180

I am not pointing to any Eigen3 directory in either of the CMake scripts.

EduFdez commented 6 years ago

@karnikram the calibration project does not crash anymore if you compile PCL with -std=c++11 as I said in my previous post.

karnikram commented 6 years ago

But unfortunately I'm not able to compile PCL 1.8 on my system :/ The process takes up all of my 4gb RAM and keeps getting killed. So I'm trying to find a fix with the binaries itself.

I got another solution, that was posted here just now.

But removing add_definitions(${PCL_DEFINITIONS}) causes a new segmentation fault. This time it's related to the mrpt libraries? I get this when I load the rawlog file:

Thread 1 "autocalib_senso" received signal SIGSEGV, Segmentation fault.
__GI___libc_free (mem=0x21) at malloc.c:3103
3103    malloc.c: No such file or directory.
(gdb) bt
#0  __GI___libc_free (mem=0x21) at malloc.c:3103
#1  0x00007ffff7af5a01 in Eigen::MatrixBase<Eigen::Matrix<float, -1, -1, 1, -1, -1> >::setSize(unsigned long, unsigned long) () from /home/karnik/tools/mrpt/build/lib/libmrpt-obs.so.1.9
#2  0x00007ffff7ae3db2 in mrpt::obs::CObservation3DRangeScan::serializeFrom(mrpt::serialization::CArchive&, unsigned char) () from /home/karnik/tools/mrpt/build/lib/libmrpt-obs.so.1.9
#3  0x00007ffff7499aa9 in mrpt::serialization::CArchive::internal_ReadObject(mrpt::serialization::CSerializable*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, signed char) () from /home/karnik/tools/mrpt/build/lib/libmrpt-serialization.so.1.9
#4  0x00007ffff7499ef6 in mrpt::serialization::CArchive::operator>>(std::shared_ptr<mrpt::serialization::CSerializable>&) () from /home/karnik/tools/mrpt/build/lib/libmrpt-serialization.so.1.9
#5  0x000055555559040e in CObservationTreeModel::CObservationTreeModel (this=0x55555739abb0, 
    rawlog_filename="/home/karnik/dataset/rgbd_1_2_2016-11-29_15h53m21s.rawlog", parent=0x5555568d49b0)
    at /home/karnik/code/autocalib-sensor-extrinsics/gui/observation_tree/CObservationTreeModel.cpp:32
#6  0x000055555557f736 in CMainWindow::loadRawlog (this=0x7fffffffe610)
    at /home/karnik/code/autocalib-sensor-extrinsics/gui/CMainWindow.cpp:107
#7  0x00005555555a3f46 in CMainWindow::qt_static_metacall (_o=0x7fffffffe610, 
    _c=QMetaObject::InvokeMetaMethod, _id=2, _a=0x7fffffffd920)
    at /home/karnik/code/autocalib-sensor-extrinsics/build/gui/moc_CMainWindow.cpp:89

Contents of ${PCL_DEFINITIONS} : -march=native -msse4.2 -mfpmath=sse -DDISABLE_ENSENSO-DDISABLE_DAVIDSDK-DDISABLE_DSSDK-DDISABLE_PCAP-DDISABLE_PNG-DDISABLE_LIBUSB_1_0-DFLANN_STATIC-DDISABLE_RSSDK-Dqh_QHpointer

EduFdez commented 6 years ago

I think that the main problem that causes the SegFault is that the binaries of PCL (and its dependencies) are compiled with a c++ standard previous to c++11. Compiling PCL with c++11 seems to solve the problem. But another option may be to remove SET(CMAKE_CXX_STANDARD 17) from your project.

jlblancoc commented 6 years ago

However, the new mrpt2 requires c++17... :-(

There are options, anyway (always there is a solution ;-):

Et voilà!

jolting commented 6 years ago

Are you using 16.04 or 18.04? I think 18.04 should work.

I backported 1.8 to 16.04. Maybe that will work.

https://launchpad.net/~jolting/+archive/ubuntu/backport-mrpt

FYI, the Ubuntu build farm is free for open source. I've never seen PCL fail to build on their farm.

https://opensourcehacker.com/2013/03/20/how-to-backport-packages-on-ubuntu-linux/

karnikram commented 6 years ago

@EduFdez, @jlblancoc, @jolting

I rebuilt mrpt in Debug mode, with debugging symbols turned on, and now somehow everything's working fine.

Earlier, removing ${PCL_DEFINITIONS} (-march=native option) stopped causing the PCL SegFault but it was causing a different SegFault in mrpt::obs::CObservation3DRangeScan::serializeFrom(mrpt::serialization::CArchive&, unsigned char) () from /home/karnik/tools/mrpt/build/lib/libmrpt-obs.so.1.9

Now after rebuilding mrpt in Debug mode, this SegFault too doesn't occur anymore, and I can run everything properly.

(I'm using PCL 1.8 binaries on 18.04)

EduFdez commented 6 years ago

That's great! Now you can continue working on the project, and we may try later to solve the problem to build in Release mode.

EduFdez commented 6 years ago

Hi @karnikram When you call a function that is time consuming, like CPlaneMatchingConfig::startCalib() you should call it in another thread, so that you do not lock the GUI, and the user can still visualize the rawlog content and the progress of the calibration. Check out the method mrpt::system::createThreadFromObjectMethod

karnikram commented 6 years ago

I believe mrpt::system::createThreadFromObjectMethod and threads.h have been removed from master. Shall I do it with std::thread?

jlblancoc commented 6 years ago

Yes, please use std::thread, that's much better since it's standard ;-)

EduFdez commented 6 years ago

Hi @karnikram There is a bit less than 3 weeks for the GSoC conclusion. To sum up about the progress so far:

You almost have it @karnikram , just a final push is needed!