MacOS crash # "on OSX OpenCV allows rendering only in main thread"?

[BigJohnn]

Hello!

After modified some configs, it was build-succeed on my macOS Catalina, but...

/Users/bigjohhn/Codes/ORB_SLAM3/Examples/Monocular-Inertial/mono_inertial_euroc /Users/bigjohhn/Codes/ORB_SLAM3/Vocabulary/ORBvoc.txt /Users/bigjohhn/Codes/ORB_SLAM3/Examples/Monocular-Inertial/EuRoC.yaml /Users/bigjohhn/Codes/ORB_SLAM3/evaluation/MH01 /Users/bigjohhn/Codes/ORB_SLAM3/Examples/Monocular-Inertial/EuRoC_TimeStamps/MH01.txt dataset-MH01_monoi

num_seq = 1
file name: dataset-MH01_monoi
Loading images for sequence 0...xxx/Users/bigjohhn/Codes/ORB_SLAM3/Examples/Monocular-Inertial/EuRoC_TimeStamps/MH01.txt
LOADED!
Loading IMU for sequence 0...LOADED!

ORB-SLAM3 Copyright (C) 2017-2020 Carlos Campos, Richard Elvira, Juan J. Gómez, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
ORB-SLAM2 Copyright (C) 2014-2016 Raúl Mur-Artal, José M.M. Montiel and Juan D. Tardós, University of Zaragoza.
This program comes with ABSOLUTELY NO WARRANTY;
This is free software, and you are welcome to redistribute it
under certain conditions. See LICENSE.txt.

Input sensor was set to: Monocular-Inertial

Loading ORB Vocabulary. This could take a while...
Vocabulary loaded!

Creation of new map with id: 0
Creation of new map with last KF id: 0
Seq. Name: 

Camera Parameters: 
- Camera: Pinhole
- fx: 458.65399169921875
- fy: 457.29598999023438
- cx: 367.21499633789062
- cy: 248.375
- k1: -0.28340810537338257
- k2: 0.073959067463874817
- p1: 0.00019359000725671649
- p2: 1.7618711353861727e-05
- fps: 20
- color order: RGB (ignored if grayscale)

ORB Extractor Parameters: 
- Number of Features: 1000
- Scale Levels: 8
- Scale Factor: 1.2000000476837158
- Initial Fast Threshold: 20
- Minimum Fast Threshold: 7

Left camera to Imu Transform (Tbc): 
[0.014865543, -0.99988091, 0.004140297, -0.021640146;
 0.99955726, 0.014967213, 0.02571553, -0.064676985;
 -0.025774436, 0.0037561883, 0.99966073, 0.0098107308;
 0, 0, 0, 1]

IMU frequency: 200 Hz
IMU gyro noise: 0.00016999999934341758 rad/s/sqrt(Hz)
IMU gyro walk: 1.9392999092815444e-05 rad/s^2/sqrt(Hz)
IMU accelerometer noise: 0.0020000000949949026 m/s^2/sqrt(Hz)
IMU accelerometer walk: 0.0030000000260770321 m/s^3/sqrt(Hz)
2021-06-02 17:36:53.555 mono_inertial_euroc[84847:340770] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'nextEventMatchingMask should only be called from the Main Thread!'
*** First throw call stack:
(
    0   CoreFoundation                      0x00007fff35249be7 __exceptionPreprocess + 250
    1   libobjc.A.dylib                     0x00007fff6e04b5bf objc_exception_throw + 48
    2   AppKit                              0x00007fff32441b65 +[NSEvent _discardTrackingAndCursorEventsIfNeeded] + 0
    3   libpangolin.dylib                   0x0000000114c2f978 +[PangolinNSApplication run_step] + 72
    4   libpangolin.dylib                   0x0000000114c29678 _ZN8pangolin9OsxWindowC2ERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEiib + 312
    5   libpangolin.dylib                   0x0000000114c2950a _ZN8pangolin9OsxWindowC1ERKNSt3__112basic_stringIcNS1_11char_traitsIcEENS1_9allocatorIcEEEEiib + 58
    6   libpangolin.dylib                   0x0000000114c29486 _ZN8pangolin22CreateOsxWindowAndBindENSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEiib + 102
    7   libpangolin.dylib                   0x0000000114c2b55d _ZZN8pangolin24RegisterOsxWindowFactoryEvEN16OsxWindowFactory4OpenERKNS_3UriE + 493
    8   libpangolin.dylib                   0x0000000114a70a62 _ZN8pangolin15FactoryRegistryINS_15WindowInterfaceEE4OpenERKNS_3UriE + 194
    9   libpangolin.dylib                   0x0000000114a6fe94 _ZN8pangolin19CreateWindowAndBindENSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEEiiRKNS_6ParamsE + 996
    10  libORB_SLAM3.dylib                  0x000000010cd11225 _ZN9ORB_SLAM36Viewer3RunEv + 149
    11  libORB_SLAM3.dylib                  0x000000010cb1597e _ZNSt3__1L14__thread_proxyINS_5tupleIJNS_10unique_ptrINS_15__thread_structENS_14default_deleteIS3_EEEEMN9ORB_SLAM36ViewerEFvvEPS8_EEEEEPvSD_ + 62
    12  libsystem_pthread.dylib             0x00007fff6f3f7109 _pthread_start + 148
    13  libsystem_pthread.dylib             0x00007fff6f3f2b8b thread_start + 15
)
libc++abi.dylib: terminating with uncaught exception of type NSException

Process finished with exit code 6

mptViewer = new thread(&Viewer::Run, mpViewer);

Any advice? Thx!

---

Solved: 1.In System.cc, annotate this line. // mptViewer = new thread(&Viewer::Run, mpViewer); and in System.h, write some "getViewer()" method. 2.In mono_inertial_euroc.cc :

auto run = [&] () {
    int proccIm=0;
    ...
    SLAM.Shutdown();
  };

new thread(run);

SLAM.getViewer()->Run();

...

Enjoy!

[sjulier]

Both the Mac and Windows have a limitation that the GUI can only run in the main thread. Linux does not have this issue.

I'm not aware that anybody has created a specific ORB-SLAM3 fix. However, we use a fork of ORB-SLAM2 for teaching and this fixes the issue (it also works under Linux as well):

https://github.com/skylook/ORB_SLAM2_MacOS

The main changes is that the image loading is now carried out in a separate thread (lambda functions in the different examples applications). There are some changes to the SLAM viewer as well.

[Fralalala]

@BigJohnn for the portion that we need to copy the following code in mono_inertial_euroc.cc:

auto run = [&] () {
    int proccIm=0;
    ...
    SLAM.Shutdown();
  };

new thread(run);

SLAM.getViewer()->Run();

...

Where do we place the code?

[BigJohnn]

Just before this comment: L235: // Save camera trajectory

@BigJohnn for the portion that we need to copy the following code in mono_inertial_euroc.cc:

auto run = [&] () {
    int proccIm=0;
    ...
    SLAM.Shutdown();
  };

new thread(run);

SLAM.getViewer()->Run();

...

Where do we place the code?

Just before this comment: L235: // Save camera trajectory