Crash in demo app when doing 3D reconstruction with two FLIR cameras (Windows)

[mlagerberg]

Posting rules

1. Duplicated posts will not be answered. Check the FAQ section, other GitHub issues, and general documentation before posting. E.g., low-speed, out-of-memory, output format, 0-people detected, installation issues, ...).
2. Fill the Your System Configuration section (all of it or it will not be answered!) if you are facing an error or unexpected behavior. Feature requests or some other type of posts might not require it.
3. No questions about training or 3rd party libraries:
   • OpenPose only implements testing.
   • Caffe errors/issues, check Caffe documentation.
   • CUDA check failed errors: They are usually fixed by re-installing CUDA, then re-installing the proper cuDNN version, and then re-compiling (or re-installing) OpenPose. Otherwise, check for help in CUDA forums.
   • OpenCV errors: Install the default/pre-compiled OpenCV or check for online help.
4. Set a proper issue title: add the Ubuntu/Windows keyword and be specific (e.g., do not call it: Error).
5. Only English comments. Posts which do not follow these rules will be ignored, closed, or reported with no further clarification.

Issue Summary

Running the demo executable crashes with an assertion error: OpenCV(4.0.1) c:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\matrix.cpp:406: error: (-215:Assertion failed) m.dims >= 2 in function 'cv::Mat::Mat

Executed Command (if any)

Note: add --logging_level 0 --disable_multi_thread to get higher debug information.

bin\OpenPoseDemo.exe --flir_camera --3d --number_people_max 1 --output_resolution "1024x768" --logging_level 0 --disable_multi_thread

OpenPose Output (if any)

*** IMAGE ACQUISITION ***

Auto-detecting all available GPUs... Detected 1 GPU(s), using 1 of them starting at GPU 0.

Error occurred on a thread. OpenPose closed all its threads and then propagated the error to the main thread. Error description:

OpenCV(4.0.1) c:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\matrix.cpp:406: error: (-215:Assertion failed) m.dims >= 2 in function 'cv::Mat::Mat'

Coming from:
- D:\Users\gines\Dropbox\Perceptual_Computing_Lab\openpose\openpose\src\openpose\3d\poseTriangulation.cpp:op::triangulate():151
- D:\Users\gines\Dropbox\Perceptual_Computing_Lab\openpose\openpose\src\openpose\3d\poseTriangulation.cpp:op::triangulateWithOptimization():302
- D:\Users\gines\Dropbox\Perceptual_Computing_Lab\openpose\openpose\src\openpose\3d\poseTriangulation.cpp:op::reconstructArrayThread():448
- D:\Users\gines\Dropbox\Perceptual_Computing_Lab\openpose\openpose\src\openpose\3d\poseTriangulation.cpp:op::PoseTriangulation::reconstructArray():523
- D:\Users\gines\Dropbox\Perceptual_Computing_Lab\openpose\openpose\include\openpose/3d/wPoseTriangulation.hpp:op::WPoseTriangulation<class std::shared_ptr<class std::vector<class std::shared_ptr<struct op::Datum>,class std::allocator<class std::shared_ptr<struct op::Datum> > > > >::work():112
- D:\Users\gines\Dropbox\Perceptual_Computing_Lab\openpose\openpose\include\openpose/thread/worker.hpp:op::Worker<class std::shared_ptr<class std::vector<class std::shared_ptr<struct op::Datum>,class std::allocator<class std::shared_ptr<struct op::Datum> > > > >::checkAndWork():93
- [All threads closed and control returned to main thread]

Errors (if any)

\matrix.cpp:406: error: (-215:Assertion failed) m.dims >= 2 in function 'cv::Mat::Mat'

Type of Issue

• Execution error

Your System Configuration

1. Whole console output (if errors appeared), paste the error to PasteBin and then paste the link here: LINK

https://pastebin.com/DYWFHrjC

2. OpenPose version: Latest GitHub code? Or specific commit (e.g., d52878f)? Or specific version from Release section (e.g., 1.2.0)?

Latest Release: 1.5.0.

3. General configuration:

   • Installation mode: downloaded OpenPoseDemo.
   • Operating system (lsb_release -a in Ubuntu): Windows
   • Operating system version (e.g., Ubuntu 16, Windows 10, ...): Windows 10 Pro N 64bit
   • Release or Debug mode? (by default: release): Release
   • Compiler (gcc --version in Ubuntu or VS version in Windows): 5.4.0, ... (Ubuntu); VS2015 Enterprise Update 3, VS2017 community, ... (Windows); ...? --

4. Non-default settings:

   • 3-D Reconstruction module added? (by default: no): yes
   • Any other custom CMake configuration with respect to the default version? (by default: no): no (default from OpenPose: gpu, python, flir, 3d)

5. 3rd-party software:

   • Caffe version: Default from OpenPose
   • CMake version (cmake --version in Ubuntu): 3.14.3
   • OpenCV version: pre-compiled apt-get install libopencv-dev (only Ubuntu); OpenPose default

6. If GPU mode issue:

   • CUDA version (cat /usr/local/cuda/version.txt in most cases): 10.1
   • cuDNN version: ?
   • GPU model (nvidia-smi in Ubuntu): NVidia GeForce GTX 1050 Ti

7. If Windows system:

   • Portable demo

[mlagerberg]

It appears that, when doing 3D-reconstruction, the argument --frame_undistort true must always be provided. When missing, the spinnakerWrapper (line 880) skips reading the camera parameter files, including the extrinsic matrices needed for reconstruction. Which makes sense, I guess, since 3D-reconstruction is not possible without undistorting the frames.

May I suggest enabling the --frame_undistort flag automatically when the --3d flag is present, or to document this behaviour? I remember the flag not being required in version 1.4.0, which makes it confusing.

[gineshidalgo99]

Sorry! All the 3D module was highly improved for 1.5 with respect to 1.4, that is why a few parameters unfortunately changed :(

However, you are right, OpenPose should report this error, the fact that OpenCV is the one reporting it means that OpenPose did not catch it. I have fixed it for the latest version. Thanks for the catch!

Let me know any further issue with it.