Closed antithing closed 2 years ago
Hello, you indeed need to include a camera calibration in your data recording folder. To do that, put in your output/
folder a file parameters.txt
OR calibration.json
. The format of parameters.txt
is given here, and an example calibration.json
looks like [1]. At the very least you need to include the focal length parameters, principal points, and the two imuToCamera
matrices. For better results you also need to specify a camera model and corresponding distortion parameters for both cameras, or otherwise the pinhole model gets used. While you could create the calibration file manually, we also provide some helper code for converting input and output from two calibration tools:
data.avi
, data2.avi
, and data.jsonl
to format Kalibr understands and perform the calibration, but you need to manually convert the output into a calibration file that HybVIO understands.
data.avi
, data2.avi
, and data.jsonl
and produces a ready-to-use calibration.json
.The file cmd.json
is not required — it can be used for setting defaults for visualizations and such. If you created that file you should probably remove it because it can crash the program if formatted incorrectly. Also in your example line main.exe -i=output p -useStereo
you must put a dash before the p
parameter: -p
.
Thanks for reporting the issue, we will try to improve the documentation regarding this. Please ask if you have further questions.
[1]: calibration.txt The extension has been changed from .json
to .txt
due to a GitHub upload limitation.
Thank you! I will take a look at this. One more question, I would like to test this system online, with a zed2 sensor, do you have any examples of running online?
Thanks again!
We do not have any online examples in this repo.
Hi, thank you for the info. i now have a calibration.json file, and running the application with the file gives me the following error:
D:\tracking_2022\HybVIO\build\Release>main.exe -i=D:\\tracking_2022\\HybVIO\\build\\Release\\output
loguru.cpp:610 arguments: odometry
loguru.cpp:613 Current dir: D:\tracking_2022\HybVIO\build\Release
loguru.cpp:615 stderr verbosity: 1
loguru.cpp:616 -----------------------------------
The system cannot find the path specified.
Failed to extract fps information from video. fpsText:
The system cannot find the path specified.
Failed to extract resolution information from video. resolutionText:
api.cpp:88 New tracking session, input resolution 600 x 176
api.cpp:94 Focal length is set to -1 (or 534.486, 534.92), principal point (635.251, 351.015)
api.cpp:886 Initializing CPU image processing
feature_detector.cpp:602 initialized feature detector (CPU-fallback)
image.cpp:354 input is BGR/RGB(A), 1280 x 720. Converting color -> gray
allocator.hpp:39 increased Allocator capacity 0 -> 5
api.cpp:927 input -> color frame swizzle: rgb1
allocator.hpp:39 increased Allocator capacity 0 -> 5
sample_sync.cpp:310 Camera frame #1 with large time difference to leader: (0.0104)s
allocator.hpp:39 increased Allocator capacity 0 -> 5
allocator.hpp:39 increased Allocator capacity 0 -> 5
allocator.hpp:39 increased Allocator capacity 0 -> 5
allocator.hpp:39 increased Allocator capacity 0 -> 5
ERROR: assertion rowWidthPixels > 0 failed in accelerated::cpu::Image::createReference (D:\tracking_2022\HybVIO\3rdparty\mobile-cv-suite\accelerated-arrays\src\cpu\image.cpp:220)
My video files are data.avi and data2.avi and do seem to be loaded okay. What might be happening here?
Thanks again.
Sounds like it doesn't find the video file or program ffprobe
. What happens if you run ffprobe D:\tracking_2022\HybVIO\build\Release\output\data.avi
on command line?
Ah, i get:
'ffprobe' is not recognized as an internal or external command, operable program or batch file.
is that part of ffmpeg? Should the ffprobe.exe be in the same dir as the main.exe file?
Downloaded and added ffprobe.exe. running from the same dir as main.exe gives me:
Input #0, avi, from 'D:\tracking_2022\HybVIO\build\Release\output\data.avi':
Duration: 00:00:17.55, start: 0.000000, bitrate: 82200 kb/s
Stream #0:0: Video: mjpeg (Baseline) (MJPG / 0x47504A4D), yuvj420p(pc, bt470bg/unknown/unknown), 1280x720 [SAR 1:1 DAR 16:9], 82264 kb/s, 60 fps, 60 tbr, 60 tbn, 60 tbc
Yes it's part of ffmpeg. It should be on path. You might need to modify this file, so it works on Windows: https://github.com/SpectacularAI/HybVIO/blob/main/src/commandline/videoutil.cpp
Okay i have added ffprobe, and I have validated the frames by viewing with opencv here:
the frames look good, so the video is loading. I have added a std::cout for
std::cout << frames[0].t << " " << frames[0].intrinsic.focalLengthX << " " <<
firstImage.width << " " <<
firstImage.height << " " << std::endl;
here:
So I can see the data is loaded as well.
This gives me:
loguru.cpp:610 arguments: odometry
loguru.cpp:613 Current dir: D:\tracking_2022\HybVIO\build\Release
loguru.cpp:615 stderr verbosity: 1
loguru.cpp:616 -----------------------------------
api.cpp:88 New tracking session, input resolution 1270 x 720
api.cpp:94 Focal length is set to -1 (or 534.486, 534.92), principal point (635.251, 351.015)
1.64079e+09 -1 1280 720
api.cpp:886 Initializing CPU image processing
feature_detector.cpp:602 initialized feature detector (CPU-fallback)
image.cpp:354 input is BGR/RGB(A), 1280 x 720. Converting color -> gray
allocator.hpp:39 increased Allocator capacity 0 -> 5
api.cpp:927 input -> color frame swizzle: rgb1
allocator.hpp:39 increased Allocator capacity 0 -> 5
1.64079e+09 -1 1280 720
sample_sync.cpp:310 Camera frame #1 with large time difference to leader: (0.0104)s
allocator.hpp:39 increased Allocator capacity 0 -> 5
allocator.hpp:39 increased Allocator capacity 0 -> 5
allocator.hpp:39 increased Allocator capacity 0 -> 5
allocator.hpp:39 increased Allocator capacity 0 -> 5
1.64079e+09 -1 1280 720
ERROR: assertion rowWidthPixels > 0 failed in accelerated::cpu::Image::createReference (D:\tracking_2022\HybVIO\3rdparty\mobile-cv-suite\accelerated-arrays\src\cpu\image.cpp:220)
It seems to be loading some frames and then failing. Is there anything I can look at to resolve this?
Thanks again for taking the time to help me out!
Hi, and thank you for making tis code available. I am building it on windows desktop, visual studio 2019, and after a day or so of tweaking, I have it compiled.
However, when I run
main.exe -i=output p -useStereo
The application does not start. ("output" is the dir containing the csv and video files data.csv, data.avi., data2.avi)I think this is because i am not passing in parameters correctly.
I am confused as to how the parameter json is created. The system looks for:
std::ifstream cmdParametersFile("../data/cmd.json");
But how do I create this file and pass in the camera intrinsics etc?
(I am using Zed2 data recorded with the zed capture application from your readme)
Thank you!