search

tum-vision / tandem

[CoRL 21'] TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo
912 stars 146 forks source link

not able to open "tandem/cva_mvsnet/model.pt" #15

Closed PeterFWS closed 2 years ago

PeterFWS commented 2 years ago

Thanks for the opensourcing code. It seems the model.pt is not provided for running your system on euroc dataset.

After running the command for the given dataset euroc, the system seems to initialize successfully but is not able to open the "model.pt" using cva_mvsnet. The flag "_mvsnetfolder=Folder that contains the exported MVSNet." is not very clearly explained. Following is my log from the terminal:

~/tandem/tandem$ ./build/bin/tandem_dataset preset=dataset result_folder=./results files=/home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/images calib=/home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/camera.txt mvsnet_folder=/home/shu/Examples/tandem/cva_mvsnet mode=1

loading data from /home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/images! loading calibration from /home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/camera.txt! Loading MVSNet from /home/shu/Examples/tandem/cva_mvsnet/! PHOTOMETRIC MODE WITHOUT CALIBRATION!

=============== TANDEM Settings: =============== Setting 'dataset':

Reading Calibration from file /home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/camera.txt ... found! Input resolution: 640 480 In: 436.225021 436.225021 310.162763 256.946041 0.000000 Out: Rectify Crop Output resolution: 640 480 finding CROP optimal new model! initial range: x: -0.7181 - 0.7613; y: -0.5949 - 0.5141! iteration 00001: range: x: -0.7145 - 0.7575; y: -0.5949 - 0.5141! iteration 00002: range: x: -0.7109 - 0.7537; y: -0.5949 - 0.5141! iteration 00003: range: x: -0.7109 - 0.7537; y: -0.5919 - 0.5115! iteration 00004: range: x: -0.7109 - 0.7537; y: -0.5890 - 0.5090! iteration 00005: range: x: -0.7109 - 0.7537; y: -0.5890 - 0.5090!

Rectified Kamera Matrix: 436.27 0 310.165 0 436.28 256.95 0 0 1

NO PHOTOMETRIC Calibration! Reading Photometric Calibration from file PhotometricUndistorter: Could not open file! set timestamps and exposures to zero! set EXPOSURES to zero! got 2778 images and 0 timestamps and 0 exposures.! ImageFolderReader: got 2778 files in /home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/images! using pyramid levels 0 to 3. coarsest resolution: 80 x 60!

----DRMVSNET Initalizing fusion----

----DRMVSNET Initalizing fusion done---- DrMvsnet torch::cuda::is_vailable == true --> seems good terminate called after throwing an instance of 'c10::Error' what(): open file failed, file path: /home/shu/Examples/tandem/cva_mvsnet/model.pt Exception raised from FileAdapter at ../caffe2/serialize/file_adapter.cc:11 (most recent call first): frame #0: c10::Error::Error(c10::SourceLocation, std::cxx11::basic_string<char, std::char_traits, std::allocator >) + 0x69 (0x7f8ea507c1d9 in /home/shu/Examples/libtorch/lib/libc10.so) frame #1: c10::detail::torchCheckFail(char const, char const, unsigned int, std::cxx11::basic_string<char, std::char_traits, std::allocator > const&) + 0xd2 (0x7f8ea5078812 in /home/shu/Examples/libtorch/lib/libc10.so) frame #2: caffe2::serialize::FileAdapter::FileAdapter(std::cxx11::basic_string<char, std::char_traits, std::allocator > const&) + 0x1ff (0x7f8ea7ddbc0f in /home/shu/Examples/libtorch/lib/libtorch_cpu.so) frame #3: torch::jit::load(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, c10::optional, std::unordered_map<std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::cxx11::basic_string<char, std::char_traits, std::allocator >, std::hash<std::cxx11::basic_string<char, std::char_traits, std::allocator > >, std::equal_to<std::cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::cxx11::basic_string<char, std::char_traits, std::allocator > const, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > >&) + 0x40 (0x7f8ea92e4b50 in /home/shu/Examples/libtorch/lib/libtorch_cpu.so) frame #4: torch::jit::load(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, c10::optional) + 0x6f (0x7f8ea92e4cdf in /home/shu/Examples/libtorch/lib/libtorch_cpu.so) frame #5: + 0x16b827 (0x55a76f9e3827 in ./build/bin/tandem_dataset) frame #6: + 0x163482 (0x55a76f9db482 in ./build/bin/tandem_dataset) frame #7: + 0x3a6bc (0x55a76f8b26bc in ./build/bin/tandem_dataset) frame #8: + 0x2980d (0x55a76f8a180d in ./build/bin/tandem_dataset) frame #9: + 0xbd6df (0x7f8e526046df in /usr/lib/x86_64-linux-gnu/libstdc++.so.6) frame #10: + 0x76db (0x7f8ea55006db in /lib/x86_64-linux-gnu/libpthread.so.0) frame #11: clone + 0x3f (0x7f8e51cc171f in /lib/x86_64-linux-gnu/libc.so.6)

Aborted (core dumped)

pankhurivanjani commented 2 years ago

~/tandem/tandem$ ./build/bin/tandem_dataset preset=dataset result_folder=./results files=/home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/images calib=/home/shu/Downloads/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/camera.txt mvsnet_folder=/home/shu/Examples/tandem/cva_mvsnet mode=1

There seems to be some typo with readme so the model has been provided but it is there in this path

tandem/tandem/exported/tandem

This is my output:

(tandem-exp) vanjani@port-2118:~/tandem/tandem/build$ ./bin/tandem_dataset preset=dataset result_folder=/home/vanjani/results files=/home/vanjani/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/images calib=/home/vanjani/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/camera.txt mvsnet_folder=/home/vanjani/tandem/tandem/exported/tandem mode=1
loading data from /home/vanjani/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/images!
loading calibration from /home/vanjani/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/camera.txt!
Loading MVSNet from /home/vanjani/tandem/tandem/exported/tandem/!
PHOTOMETRIC MODE WITHOUT CALIBRATION!

=============== TANDEM Settings: ===============
    Setting 'dataset':
    - no real-time enforcing
    - 2000 active points
    - 5-7 active frames
    - 1-6 LM iteration each KF
    - TSDF fusion: yes
    - dense tracking on cpu (step=1)
    - Pangolin
      - Fullscreen: 0
      - Mesh: 1
      - Smaller Images: 1

Reading Calibration from file /home/vanjani/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/camera.txt ... found!
Input resolution: 640 480
In: 436.225021 436.225021 310.162763 256.946041 0.000000
Out: Rectify Crop
Output resolution: 640 480
finding CROP optimal new model!
initial range: x: -0.7181 - 0.7613; y: -0.5949 - 0.5141!
iteration 00001: range: x: -0.7145 - 0.7575; y: -0.5949 - 0.5141!
iteration 00002: range: x: -0.7109 - 0.7537; y: -0.5949 - 0.5141!
iteration 00003: range: x: -0.7109 - 0.7537; y: -0.5919 - 0.5115!
iteration 00004: range: x: -0.7109 - 0.7537; y: -0.5890 - 0.5090!
iteration 00005: range: x: -0.7109 - 0.7537; y: -0.5890 - 0.5090!

Rectified Kamera Matrix:
 436.27       0 310.165
      0  436.28  256.95
      0       0       1

NO PHOTOMETRIC Calibration!
Reading Photometric Calibration from file 
PhotometricUndistorter: Could not open file!
set timestamps and exposures to zero!
set EXPOSURES to zero!
got 2778 images and 0 timestamps and 0 exposures.!
ImageFolderReader: got 2778 files in /home/vanjani/euroc_tandem_format_1.1.beta/euroc_tandem_format/V1_01_easy/images!
using pyramid levels 0 to 3. coarsest resolution: 80 x 60!

----DRMVSNET Initalizing fusion----

----DRMVSNET Initalizing fusion done----
DrMvsnet torch::cuda::is_vailable == true --> seems good
View Num: 7, ref index: 5
[W BinaryOps.cpp:467] Warning: floor_divide is deprecated, and will be removed in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values.
To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). (function operator())
Correctness:
    Depth correct     : 1, error: 0.000665135
    Confidence correct: 1, error: 0.000441843
Correctness:
    Depth correct     : 1, error: 0.000665166
    Confidence correct: 1, error: 0.000441809
Correctness:
    Depth correct     : 1, error: 0.000665188
    Confidence correct: 1, error: 0.000441806
Correctness:
    Depth correct     : 1, error: 0.000665184
    Confidence correct: 1, error: 0.000441807
Correctness:
    Depth correct     : 1, error: 0.000665187
    Confidence correct: 1, error: 0.000441804
Correctness:
    Depth correct     : 1, error: 0.000665187
    Confidence correct: 1, error: 0.000441807
Correctness:
    Depth correct     : 1, error: 0.00066518
    Confidence correct: 1, error: 0.000441806
Correctness:
    Depth correct     : 1, error: 0.000665187
    Confidence correct: 1, error: 0.000441806
Correctness:
    Depth correct     : 1, error: 0.000665185
    Confidence correct: 1, error: 0.000441804
Performance:
    CallAsync     : 8.25375 ms
    Ready         : 0 ms
    GetResult     : 378.74 ms
All looks good!
INITIALIZE FROM INITIALIZER (2043 pts)!
lkskstlr commented 2 years ago

@pankhurivanjani Thank you for the helpful comment :)

@PeterFWS I will close this issue, feel free to re-open if necessary.