void dso::CoarseInitializer::makeNN(): Assertion `ret_index[0]>=0 && ret_index[0] < numPoints[lvl+1]' failed

[adynathos]

I am trying to run DSO on the Cityscapes video dataset and encountering an error. I would like to ask for help with understanding or fixing it.

The program crashes instantly with the following error:

FullSystem/CoarseInitializer.cpp:1025: void dso::CoarseInitializer::makeNN(): Assertion `ret_index[0]>=0 && ret_index[0] < numPoints[lvl+1]' failed.

I use the following presets and modes:

./build/bin/dso_dataset \
    files=/home/adynathos/dev/phd/dsets/cityscapes/leftImg8bit/demoVideo/stuttgart_00 \
    calib=/home/adynathos/dev/phd/dsets/cityscapes/leftImg8bit/demoVideo/dso_calib_file.txt \
    preset=0 \
    mode=1

Camera calibration file:

2262.52     2265.3017905988554  1096.98 513.137 0
2048    1024
none
2048    1024

Complete output:

loading data from /home/adynathos/dev/phd/dsets/cityscapes/leftImg8bit/demoVideo/stuttgart_00!
loading calibration from /home/adynathos/dev/phd/dsets/cityscapes/leftImg8bit/demoVideo/dso_calib_file.txt!

=============== PRESET Settings: ===============
DEFAULT settings:
- no  real-time enforcing
- 2000 active points
- 5-7 active frames
- 1-6 LM iteration each KF
- original image resolution
==============================================
PHOTOMETRIC MODE WITHOUT CALIBRATION!
Reading Calibration from file /home/adynathos/dev/phd/dsets/cityscapes/leftImg8bit/demoVideo/dso_calib_file.txt ... found!
found PINHOLE camera model, building rectifier.
Input resolution: 2048 1024
In: 2262.520000 2265.301791 1096.980000 513.137000 0.000000
Out: No Rectification
Output resolution: 2048 1024

Rectified Kamera Matrix:
2262.52       0 1096.98
      0  2265.3 513.137
      0       0       1

NO PHOTOMETRIC Calibration!
Reading Photometric Calibration from file 
PhotometricUndistorter: Could not open file!
set timestamps and exposures to zero!
set EXPOSURES to zero!
got 599 images and 0 timestamps and 0 exposures.!
ImageFolderReader: got 599 files in /home/adynathos/dev/phd/dsets/cityscapes/leftImg8bit/demoVideo/stuttgart_00!
using pyramid levels 0 to 5. coarsest resolution: 64 x 32!
START PANGOLIN!
dso_dataset: /home/adynathos/dev/phd/slam/dso/dso_src/src/FullSystem/CoarseInitializer.cpp:1025: void dso::CoarseInitializer::makeNN(): Assertion `ret_index[0]>=0 && ret_index[0] < numPoints[lvl+1]' failed.

Thank you.

[NikolausDemmel]

Hm... it's not immediately clear what the problem might be. Can you run with successfully with the suggested TUM mono dataset?

[trocolit2]

I had this error too. It happen when I pass a smoothed image, with no edges, like this image :

[adynathos]

I did further tests and I believe the problem is related to pyramid levels and coarsest resolution.

(1) When running the TUM dataset with standard settings, the pyramid is as follows: using pyramid levels 0 to 3. coarsest resolution: 80 x 60! No crash

(2) Now I changed the camera.txt file and disabled rectification:

**0.349153000000000 0.436593000000000   0.493140000000000   0.499021000000000   0.933271000000000
1280 1024
none
1280 1024

The new image size results in the following pyramid: using pyramid levels 0 to 5. coarsest resolution: 40 x 32! Crash occurs

(3) Then I resized the images to 1396x1040 (GCD=16), which prevents that 5th level of pyramid: using pyramid levels 0 to 4. coarsest resolution: 81 x 65! No crash.

---

So the crash seems to be caused by one of those:

• pyramid levels 5 or higher
• low coarsest resolution (for example 40x32)

---

Location of the crash:

if(lvl < pyrLevelsUsed-1 )
{
    resultSet1.init(ret_index, ret_dist);
    pt = pt*0.5f-Vec2f(0.25f,0.25f);
    indexes[lvl+1]->findNeighbors(resultSet1, (float*)&pt, nanoflann::SearchParams());

    pts[i].parent = ret_index[0];
    pts[i].parentDist = expf(-ret_dist[0]*NNDistFactor);

    assert(ret_index[0]>=0 && ret_index[0] < numPoints[lvl+1]);
}
else
{
    pts[i].parent = -1;
    pts[i].parentDist = -1;
}

Variables according to the debugger:

pyrLevelsUsed = 6
ret_index = [0, 1, 47, 48, 46, 110, 2, 109, 111, 49]
numPoints = [34338, 44422, 47487, 12634, 3455, 0]

The crash happens when lvl=4 and the assert is ret_index[0] = 0 < numPoints[5] = 0.

I would be grateful for help.

[adynathos]

Setting PYR_LEVELS to 6 (as opposed to default 7) in settings.h restricts the pyramid level and prevents the crash.

[fangxu622]

@NikolausDemmel @adynathos hi ，I get a similar error.

dso_dataset: /media/fangxu/Segate3T/Linux-Proj/SLAM/dso/src/FullSystem/CoarseTracker.cpp:546: bool dso::CoarseTracker::trackNewestCoarse(dso::FrameHessian*, dso::SE3&, dso::AffLight&, int, dso::Vec5, dso::IOWrap::Output3DWrapper*): Assertion `coarsestLvl < 5 && coarsestLvl < pyrLevelsUsed' failed.
Aborted (core dumped)

Due to coarsetLvl values. When I set PYR_LEVELS to 7 or above. It will trigger some errors else. the message like this

LOG 7: 19.285 fine. Res: 1100 A, 0 L, 0 M; (0 / 0) forceDrop. a=-0.356354, b=32.152864. Window 7 (2)
dso_dataset: /usr/local/include/eigen3/Eigen/src/Core/DenseCoeffsBase.h:408: Eigen::DenseCoeffsBase<Derived, 1>::Scalar& Eigen::DenseCoeffsBase<Derived, 1>::operator[](Eigen::Index) [with Derived = Eigen::Matrix<double, 5, 1>; Eigen::DenseCoeffsBase<Derived, 1>::Scalar = double; Eigen::Index = long int]: Assertion `index >= 0 && index < size()' failed.
Aborted (core dumped)

My camera is GoPro 7. Calibrattion model is kanalla brandt.

933.591144672   939.968223442   959.50  719.50  1.3571629546076457e-02  1.7799816234275553e-01  -2.9965104051643743e-01 1.8437185124189442e-01
1920 1440
none
1920 1440

[NikolausDemmel]

To trace this you guys will have to use a debugger to see where exactly it cradhes and then try to figure out the cause.

[AI-ML-Enthusiast]

@NikolausDemmel @adynathos I have the same problem. My error is : Reading Photometric Calibration from file ../dataset/fv1/pcalib.txt Reading Vignette Image from ../dataset/fv1/vignette.png Successfully read photometric calibration! got 754 images and 754 timestamps and 754 exposures.! ImageFolderReader: got 754 files in ../dataset/fv1/images.zip! using pyramid levels 0 to 3. coarsest resolution: 240 x 135!

===============WARNING!=================== using not enough pyramid levels. Consider scaling to a resolution that is a multiple of a power of 2. OUT: Created SampleOutputWrapper START PANGOLIN! /src/FullSystem/CoarseInitializer.cpp:1025: void dso::CoarseInitializer::makeNN(): Assertion `ret_index[0]>=0 && ret_index[0] < numPoints[lvl+1]' failed. Aborted (core dumped) Anyone can help me please. My data info: Width: 1920 height: 1080, and camera.txt contains: 1.725592688 3.064300037 1.03824376 1.364821278 0 1920 1080 crop 1920 1080

But if I change crop to none then it is running but visualization is not good, something wrong

[AI-ML-Enthusiast]

Setting PYR_LEVELS to 6 (as opposed to default 7) in settings.h restricts the pyramid level and prevents the crash.

@adynathos Is there any other solution without changing the PYR_LEVELS value? Thanks

[rojones94]

Hm... it's not immediately clear what the problem might be. Can you run with successfully with the suggested TUM mono dataset?

I am trying to replicate the demo on TUM using sequence 42 (unsuccessfully). If I run with the following as my camera.txt I get the same error

""" 0.535719308086809 0.669566858850269 0.493248545285398 0.500408664348414 0.897966326944875 1280 1024 none 1280 1024 """

gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0 cmake version 3.25.0