search

tum-vision / tandem

[CoRL 21'] TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo
912 stars 146 forks source link

Couldn't load MVSNet successfully.INITIALIZE FROM INITIALIZER #41

Open pankhurivanjani opened 1 year ago

pankhurivanjani commented 1 year ago

Hello,

I get this in logs while running tandem for any dataset,

(tandem) pankhuri@pankhuri-GE76-Raider-11UH:~/tandem/tandem$ ./build/bin/tandem_dataset preset=gui result_folder=/home/pankhuri/tandem-dataset/results files=/home/pankhuri/tandem-dataset/euroc_tandem_format/V1_01_easy/images calib=/home/pankhuri/tandem-dataset/euroc_tandem_format/V1_01_easy/camera.txt mvsnet_folder=/home/pankhuri/tandem/tandem/exported/tandem mode=1
loading data from /home/pankhuri/tandem-dataset/euroc_tandem_format/V1_01_easy/images!
loading calibration from /home/pankhuri/tandem-dataset/euroc_tandem_format/V1_01_easy/camera.txt!
Loading MVSNet from /home/pankhuri/tandem/tandem/exported/tandem/!
PHOTOMETRIC MODE WITHOUT CALIBRATION!

=============== TANDEM Settings: ===============
    Setting 'gui':
    - no real-time enforcing
    - 2000 active points
    - 5-7 active frames
    - 1-6 LM iteration each KF
    - TSDF fusion: yes
    - dense tracking on cpu (step=1)
    - Pangolin
      - Fullscreen: 0
      - Mesh: 1
      - Smaller Images: 1

Reading Calibration from file /home/pankhuri/tandem-dataset/euroc_tandem_format/V1_01_easy/camera.txt ... found!
Input resolution: 640 480
In: 436.225021 436.225021 310.162763 256.946041 0.000000
Out: Rectify Crop
Output resolution: 640 480
finding CROP optimal new model!
initial range: x: -0.7181 - 0.7613; y: -0.5949 - 0.5141!
iteration 00001: range: x: -0.7145 - 0.7575; y: -0.5949 - 0.5141!
iteration 00002: range: x: -0.7109 - 0.7537; y: -0.5949 - 0.5141!
iteration 00003: range: x: -0.7109 - 0.7537; y: -0.5919 - 0.5115!
iteration 00004: range: x: -0.7109 - 0.7537; y: -0.5890 - 0.5090!
iteration 00005: range: x: -0.7109 - 0.7537; y: -0.5890 - 0.5090!

Rectified Kamera Matrix:
 436.27       0 310.165
      0  436.28  256.95
      0       0       1

NO PHOTOMETRIC Calibration!
Reading Photometric Calibration from file 
PhotometricUndistorter: Could not open file!
set timestamps and exposures to zero!
set EXPOSURES to zero!
got 2778 images and 0 timestamps and 0 exposures.!
ImageFolderReader: got 2778 files in /home/pankhuri/tandem-dataset/euroc_tandem_format/V1_01_easy/images!
using pyramid levels 0 to 3. coarsest resolution: 80 x 60!
START PANGOLIN!

----DRMVSNET Initalizing fusion----

----DRMVSNET Initalizing fusion done----
DrMvsnet torch::cuda::is_vailable == true --> seems good
View Num: 7, ref index: 5
[W BinaryOps.cpp:467] Warning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (function operator())
Correctness:
    Depth correct     : 0, error: 0.0106286
    Confidence correct: 1, error: 0.00296125
Correctness:
    Depth correct     : 0, error: 0.0106182
    Confidence correct: 1, error: 0.00295481
Correctness:
    Depth correct     : 0, error: 0.0106182
    Confidence correct: 1, error: 0.0029548
Correctness:
    Depth correct     : 0, error: 0.0106182
    Confidence correct: 1, error: 0.00295482
Correctness:
    Depth correct     : 0, error: 0.0106182
    Confidence correct: 1, error: 0.00295482
Correctness:
    Depth correct     : 0, error: 0.0106182
    Confidence correct: 1, error: 0.00295482
Correctness:
    Depth correct     : 0, error: 0.0106182
    Confidence correct: 1, error: 0.00295482
Correctness:
    Depth correct     : 0, error: 0.0106182
    Confidence correct: 1, error: 0.0029548
Correctness:
    Depth correct     : 0, error: 0.0106182
    Confidence correct: 1, error: 0.00295482
Performance:
    CallAsync     : 6.655 ms
    Ready         : 0 ms
    GetResult     : 116.791 ms
There has been an error. Do not use the model.
Couldn't load MVSNet successfully.INITIALIZE FROM INITIALIZER (2043 pts)!

TANDEM TIMING: ==================
2777 Frames (11.8 fps)
84.59ms per frame; 
234.91s total time; 
======================

DrFusion::SaveMeshToFile volume_->ExtractMesh (482 ms)
DrFusion::SaveMeshToFile mesh.SaveToFile (17763 ms)
Mesh Saving done!

I can see the 3D reconstruction but this error still exists.

I wish to ask:

  1. Is anyone else also getting the same error? How do we solve this ?
  2. Is it related to the warning that floor_divide is deprecated?
  3. I am not getting good results on other custom datasets with tandem, Can this error affect the 3D reconstruction on a custom synthetic dataset?

Thank you,

pytholic commented 1 year ago

Hello,

I get this in logs while running tandem for any dataset,

(tandem) pankhuri@pankhuri-GE76-Raider-11UH:~/tandem/tandem$ ./build/bin/tandem_dataset preset=gui result_folder=/home/pankhuri/tandem-dataset/results files=/home/pankhuri/tandem-dataset/euroc_tandem_format/V1_01_easy/images calib=/home/pankhuri/tandem-dataset/euroc_tandem_format/V1_01_easy/camera.txt mvsnet_folder=/home/pankhuri/tandem/tandem/exported/tandem mode=1
loading data from /home/pankhuri/tandem-dataset/euroc_tandem_format/V1_01_easy/images!
loading calibration from /home/pankhuri/tandem-dataset/euroc_tandem_format/V1_01_easy/camera.txt!
Loading MVSNet from /home/pankhuri/tandem/tandem/exported/tandem/!
PHOTOMETRIC MODE WITHOUT CALIBRATION!

=============== TANDEM Settings: ===============
  Setting 'gui':
  - no real-time enforcing
  - 2000 active points
  - 5-7 active frames
  - 1-6 LM iteration each KF
  - TSDF fusion: yes
  - dense tracking on cpu (step=1)
  - Pangolin
    - Fullscreen: 0
    - Mesh: 1
    - Smaller Images: 1

Reading Calibration from file /home/pankhuri/tandem-dataset/euroc_tandem_format/V1_01_easy/camera.txt ... found!
Input resolution: 640 480
In: 436.225021 436.225021 310.162763 256.946041 0.000000
Out: Rectify Crop
Output resolution: 640 480
finding CROP optimal new model!
initial range: x: -0.7181 - 0.7613; y: -0.5949 - 0.5141!
iteration 00001: range: x: -0.7145 - 0.7575; y: -0.5949 - 0.5141!
iteration 00002: range: x: -0.7109 - 0.7537; y: -0.5949 - 0.5141!
iteration 00003: range: x: -0.7109 - 0.7537; y: -0.5919 - 0.5115!
iteration 00004: range: x: -0.7109 - 0.7537; y: -0.5890 - 0.5090!
iteration 00005: range: x: -0.7109 - 0.7537; y: -0.5890 - 0.5090!

Rectified Kamera Matrix:
 436.27       0 310.165
      0  436.28  256.95
      0       0       1

NO PHOTOMETRIC Calibration!
Reading Photometric Calibration from file 
PhotometricUndistorter: Could not open file!
set timestamps and exposures to zero!
set EXPOSURES to zero!
got 2778 images and 0 timestamps and 0 exposures.!
ImageFolderReader: got 2778 files in /home/pankhuri/tandem-dataset/euroc_tandem_format/V1_01_easy/images!
using pyramid levels 0 to 3. coarsest resolution: 80 x 60!
START PANGOLIN!

----DRMVSNET Initalizing fusion----

----DRMVSNET Initalizing fusion done----
DrMvsnet torch::cuda::is_vailable == true --> seems good
View Num: 7, ref index: 5
[W BinaryOps.cpp:467] Warning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (function operator())
Correctness:
  Depth correct     : 0, error: 0.0106286
  Confidence correct: 1, error: 0.00296125
Correctness:
  Depth correct     : 0, error: 0.0106182
  Confidence correct: 1, error: 0.00295481
Correctness:
  Depth correct     : 0, error: 0.0106182
  Confidence correct: 1, error: 0.0029548
Correctness:
  Depth correct     : 0, error: 0.0106182
  Confidence correct: 1, error: 0.00295482
Correctness:
  Depth correct     : 0, error: 0.0106182
  Confidence correct: 1, error: 0.00295482
Correctness:
  Depth correct     : 0, error: 0.0106182
  Confidence correct: 1, error: 0.00295482
Correctness:
  Depth correct     : 0, error: 0.0106182
  Confidence correct: 1, error: 0.00295482
Correctness:
  Depth correct     : 0, error: 0.0106182
  Confidence correct: 1, error: 0.0029548
Correctness:
  Depth correct     : 0, error: 0.0106182
  Confidence correct: 1, error: 0.00295482
Performance:
  CallAsync     : 6.655 ms
  Ready         : 0 ms
  GetResult     : 116.791 ms
There has been an error. Do not use the model.
Couldn't load MVSNet successfully.INITIALIZE FROM INITIALIZER (2043 pts)!

TANDEM TIMING: ==================
2777 Frames (11.8 fps)
84.59ms per frame; 
234.91s total time; 
======================

DrFusion::SaveMeshToFile volume_->ExtractMesh (482 ms)
DrFusion::SaveMeshToFile mesh.SaveToFile (17763 ms)
Mesh Saving done!

I can see the 3D reconstruction but this error still exists.

I wish to ask:

  1. Is anyone else also getting the same error? How do we solve this ?
  2. Is it related to the warning that floor_divide is deprecated?
  3. I am not getting good results on other custom datasets with tandem, Can this error affect the 3D reconstruction on a custom synthetic dataset?

Thank you,

HI @pankhurivanjani , did you figure this out?

yahskapar commented 10 months ago

@pankhurivanjani or @pytholic, were either of you able to figure this out? Also currently stuck on trying to figure out why this, coupled with an eventual segmentation fault, is happening...

EDIT: I still get the error message regarding MVSNet, but I was able to generate a mesh regardless after getting past the segmentation fault using the advice here.