About an execution error

[Segfred]

Hello, Haarbach, I am Robert. I find your work great but I encounter a problem when running your code. I am using Mac Yosemite and following exactly your instructions about how to run the code. The thing is when I run ./ppf-reconstruction, everything is OK and I get a sparse reconstruction. But when I run ./ppf_reconstruction -dir ../samples/Bunny_Sphere/ -knn 5 -nFrames 5, the program just aborts and shows that "libc++abi.dylib: terminating with uncaught exception of type Iex::EnosysExc: Cannot initialize semaphore (Function not implemented)." I am wondering if you know how to fix that problem? With Best!

[adrelino]

Hi Robert

sorry for my late reply, I was quite busy recently.

Even though I could not yet reproduce your bug, I think we can narrow it down to what is actually failing.

The only difference between running ./ppf-reconstruction (wich defaults to -dir ../samples/Bunny_RealData/) and ./ppf_reconstruction -dir ../samples/Bunny_Sphere/ -knn 5 -nFrames 5 is that the first one (that works for you) uses real data and real depth maps grabbed from Kinect and saved in .png format and the latter one (which fails for you) uses synthetic data and generated depth maps from Blender saved in .exr format.

I also googled a little bit for your error, which hinted towards OpenCV in combination with OpenEXR loading issues.

To be sure, could you check out my code again because I added some more verbosity output before and after calls to cv::imread (which loads the .png or .exr files). If everything there works fine, you should now get the following output for each depth image loaded:

./ppf_reconstruction -dir ../samples/Bunny_Sphere/ -knn 5 -nFrames 5
 PointCloud fromDepthMap constructor
LoadingSaving::loadPointCloudFromDepthMap before cv::imread ../samples/Bunny_Sphere//depth_000000.exr
LoadingSaving::loadPointCloudFromDepthMap after cv::imread ../samples/Bunny_Sphere//depth_000000.exr     type: CV_32FC3 channels:3
DownSampled 11215->109 pts with voxelSize:0.015 minPtsPerVoxel:20

if you only see LoadingSaving::loadPointCloudFromDepthMap before cv::imread but not the log after calling the function, we know it has to do with OpenCV + OpenEXR

could you please check that for me?

Also, please send me the output of running brew info opencv maybe something changed in the brew opencv package in the meantime which caused the bug

[Segfred]

Hi, Haarbach. Thank you very much for your detailed explanation and I have retried your code in another iMac( from my laboratory). It turns out that both of your commands can run perfectly on the iMac.

Part1: on the Macbook (where the second command cannot run) But as for my own Macbook, I even cannot see the following result 「LoadingSaving::loadPointCloudFromDepthMap before cv::imread 」All I can see is 「PointCloud fromDepthMap constructor」,then the program just aborts. Besides, I installed the opencv 2.4 by source before and it exists in the usr/local/include address now, instead of in usr/local/Cellar (default address by brew install opencv). When I type brew info opencv, what I see is as below: (it seems my opencv cannot be found by this program).

cmd: brew info opencv 「homebrew/science/opencv: stable 2.4.12 (bottled), HEAD Open source computer vision library http://opencv.org/ Not installed From: https://github.com/homebrew/homebrew-science/blob/master/opencv.rb ==> Dependencies Build: cmake ✔, pkg-config ✘ Required: jpeg ✘, libpng ✘, libtiff ✘ Recommended: eigen ✔, openexr ✘, homebrew/python/numpy ✘ Optional: gstreamer ✘, jasper ✘, libdc1394 ✘, openni ✘, qt ✘, tbb ✘, vtk ✘, ffmpeg ✘ ==> Options --32-bit Build 32-bit only --c++11 Build using C++11 mode --universal Build a universal binary --with-cuda Build with CUDA support --with-ffmpeg Build with ffmpeg support --with-gstreamer Build with gstreamer support --with-jasper Build with jasper support --with-java Build with Java support --with-libdc1394 Build with libdc1394 support --with-opengl Build with OpenGL support --with-openni Build with openni support --with-qt Build the Qt4 backend to HighGUI --with-quicktime Use QuickTime for Video I/O instead of QTKit --with-tbb Enable parallel code in OpenCV using Intel TBB --with-vtk Build with vtk support --with-ximea Build with XIMEA support --without-eigen Build without eigen support --without-numpy Use a numpy you've installed yourself instead of a Homebrew-packaged numpy --without-opencl Disable GPU code in OpenCV using OpenCL --without-openexr Build without openexr support --without-python Build without Python support --without-tests Build without accuracy & performance tests --HEAD Install HEAD version AsmitadeMacBook-Pro:build asmita$ 」

In usr/local/Cellar: there exist brew-cask, cmake, eigen, free glut In usr/local/include: there exist EXTERNAL, eigen3, opencv, pcl-1.8, GL, g2o, OPENCV2

Sorry about the mess in my Macbook since I installed many libraries before. Then I switch to the iMac in my laboratory and got the following result

Part2: on the iMac (where both commands are OK) cmd: brew info opencv homebrew/science/opencv: stable 2.4.12 (bottled), HEAD Open source computer vision library http://opencv.org/ /usr/local/Cellar/opencv/2.4.12_2 (225 files, 35.6M) * Poured from bottle From: https://github.com/Homebrew/homebrew-science/blob/master/opencv.rb ==> Dependencies Build: cmake ✔, pkg-config ✘ Required: jpeg ✔, libpng ✔, libtiff ✔ Recommended: eigen ✔, openexr ✔, homebrew/python/numpy ✔ Optional: gstreamer ✘, jasper ✘, libdc1394 ✘, openni ✘, qt ✘, tbb ✔, vtk ✘, ffmpeg ✘ ==> Options --32-bit Build 32-bit only --c++11 Build using C++11 mode --universal Build a universal binary --with-cuda Build with CUDA support --with-ffmpeg Build with ffmpeg support --with-gstreamer Build with gstreamer support --with-jasper Build with jasper support --with-java Build with Java support --with-libdc1394 Build with libdc1394 support --with-opengl Build with OpenGL support --with-openni Build with openni support --with-qt Build the Qt4 backend to HighGUI --with-quicktime Use QuickTime for Video I/O instead of QTKit --with-tbb Enable parallel code in OpenCV using Intel TBB --with-vtk Build with vtk support --with-ximea Build with XIMEA support --without-eigen Build without eigen support --without-numpy Use a numpy you've installed yourself instead of a Homebrew-packaged numpy --without-opencl Disable GPU code in OpenCV using OpenCL --without-openexr Build without openexr support --without-python Build without Python support --without-tests Build without accuracy & performance tests --HEAD Install HEAD version ==> Caveats Python modules have been installed and Homebrew's site-packages is not in your Python sys.path, so you will not be able to import the modules this formula installed. If you plan to develop with these modules, please run: mkdir -p /Users/lab/Library/Python/2.7/lib/python/site-packages echo 'import site; site.addsitedir("/usr/local/lib/python2.7/site-packages")' >> /Users/lab/Library/Python/2.7/lib/python/site-packages/homebrew.pth

By the way, I am wondering have you ever thought about taking ordinary pictures instead of range images as input? And have you ever thought about model fitting (surface reconstruction) as well as texture mapping after getting the dense point cloud? Could you be so kind to tell me your recent focus?

Best Wishes! Robert

[lironesamoun]

I got the same issue concerning the command ./ppf_reconstruction -dir ../samples/Bunny_Sphere -knn 5 -nFrames 5

I got this message :

Found 42 files in ../samples/Bunny_Sphere that match the pattern: depth.png|.jpg|.tif|.exr|.bmp . first is: ../samples/Bunny_Sphere/depth_000000.exr Found 0 files in ../samples/Bunny_Sphere that match the pattern: mask.png|.jpg|.tif|.exr|.bmp . Found 1 files in ../samples/BunnySphere that match the pattern: Intrinsic.txt|.xyz . first is: ../samples/Bunny_Sphere/Intrinsic.txt Found 42 files in ../samples/BunnySphere that match the pattern: pose.txt|.xyz . first is: ../samples/Bunny_Sphere/poses_000000.txt loaded 42 poses from dir ../samples/Bunny_Sphere with prefix pose PointCloud fromDepthMap constructor LoadingSaving::loadPointCloudFromDepthMap before cv::imread ../samples/Bunny_Sphere/depth_000000.exr libc++abi.dylib: terminating with uncaught exception of type Iex::EnosysExc: Cannot initialize semaphore (Function not implemented)

Concerning the command line brew info opencv

I got

homebrew/science/opencv: stable 2.4.13 (bottled), HEAD Open source computer vision library http://opencv.org/ /usr/local/Cellar/opencv/2.4.12_2 (345 files, 108.2M) Poured from bottle on 2016-02-15 at 09:47:01 /usr/local/Cellar/opencv/2.4.13 (276 files, 35.8M) Poured from bottle on 2016-07-01 at 09:50:56 From: https://github.com/Homebrew/homebrew-science/blob/master/opencv.rb ==> Dependencies Build: cmake ✔, pkg-config ✔ Required: jpeg ✔, libpng ✔, libtiff ✔ Recommended: eigen ✔, openexr ✔, homebrew/python/numpy ✔ Optional: gstreamer ✔, jasper ✘, libdc1394 ✘, openni ✘, qt ✔, tbb ✔, vtk ✔, ffmpeg ✘

Segfred rise an interesting question ! Is it possible to the reconstruction using view from camera ? Thank

[adrelino]

Hi

The error

Iex::EnosysExc

Is defined by OpenEXR. http://stackoverflow.com/questions/22688117/writing-a-float-image-in-opencv-with-pixel-values-bigger-than-1/22689373#22689373 https://github.com/openexr/openexr/blob/master/IlmBase/Iex/IexErrnoExc.h#L129

Have you tried reinstalling opencv and openexr? brew uninstall opencv openexr && brew install openexr opencv && brew doctor Maybe something went wrong when linking them (e.g. maybe they were compiled using different compilers and std libs).

[adrelino]

By the way, I am wondering have you ever thought about taking ordinary pictures instead of range images as input? Segfred rise an interesting question ! Is it possible to the reconstruction using view from camera ? Thank

No, it is a purely geometry based approach, thus range images are used and unprojected using the pinhole camera model to get points in 3d space. If you have a stereo camera system one could use stereo vision techniques to get these 3d points from 2 RGB images, but that was out of the scope here.

And have you ever thought about model fitting () as well as texture mapping after getting the dense point cloud? Could you be so kind to tell me your recent focus?

The final result of my pipeline is just the optimized trajectory / camera pose of each frame.

The next steps in a standard scan-to-model / surface reconstruction pipeline would be to:

1. project the points from each scan into a common reference frame.
2. Build a volumetric model, e.g. using a signed distance volume
3. Extract the isocontour and build a triangle mesh using the Marching Cubes Algorithm
4. use the RGB images and some sort of ray-casting to color these triangles. Need a smart way to integrate the color of a point in 3D when seen multiple times from different angles and lightning conditions.

But this is not in my focus right now, there are already lots of solutions who deal with this.

[swframe]

I had this error and the info here helped. I thought I would add a little extra info to make it more clear how to fix the problem. If you build opencv you have to disable the posix semaphore api calls. The following page explains how to build opencv on mac os: http://www.pyimagesearch.com/2015/06/15/install-opencv-3-0-and-python-2-7-on-osx/ It gives a cmake command and you need to add "-D HAVE_POSIX_SEMAPHORES=0"