search

uzh-rpg / vimo

Visual-Inertial Model-based State and External Forces Estimator
GNU General Public License v3.0
179 stars 47 forks source link

Regd. IMU parameters used for the Blackbird dataset #1

Open karnikram opened 3 years ago

karnikram commented 3 years ago

Hi,

Thank you for releasing your code.

I've been trying to run VINS-Mono and your code on the Blackbird dataset. But there seems to be a problem with the IMU intrinsic values and the camera-IMU transform used in your config file. These values don't match what is given in their dataset (and in their IJRR paper), and when I run VINS-Mono using these values it keeps failing.

Values reported in their dataset:

T_imu_cam: [1.0, 0.0, 0.0, -0.05,
0.0, 0.0, -1.0, 0.0,
0.0, 1.0, 0.0, 0.0,
0.0, 0.0, 0.0, 1.0]

IMU.NoiseGyro: 1.2e-4 
IMU.NoiseAcc: 2.0e-3
IMU.GyroWalk: 4.7e-6 
IMU.AccWalk: 4.4e-5

Values in your config file:

T_imu_cam: [0.0, 0.0, 1.0, 0.0,
-1.0, 0.0, 0.0, 0.0,
0.0, -1.0, 0.0, 0,0,
0.0, 0.0, 0.0, 1.0]

IMU.NoiseGyro: 0.1 
IMU.NoiseAcc: 0.3
IMU.GyroWalk: 0.003
IMU.AccWalk: 0.001

Am I missing something?

juanmed commented 3 years ago

Hi @karnikram

were you able to run VINS-Mono the blackbird dataset using vimo? I am also seeing VINS-Mono fail on both star and picasso trajectories. The program outputs:

W0913 18:03:22.341883 22439 levenberg_marquardt_strategy.cc:114] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization.
[ WARN] [1599987802.399025387, 1525765667.055149141]: gyroscope bias initial calibration 0 0 0
[ INFO] [1599987802.400740441, 1525765667.055149141]: estimated scale: 0.000000
[ INFO] [1599987802.400880508, 1525765667.057669834]:  result g     0 0 0 0
[ INFO] [1599987802.400934201, 1525765667.057669834]: misalign visual structure with IMU
karnikram commented 3 years ago

Hi @juanmed,

Yes, I am able to run VINS-Mono on blackbird sequences. I've attached my config file here. The extrinsics and IMU parameters are obtained from the blackbird dataset repo, and are different from what is provided in this repo. I also throttled the image rate of blackbird from 120 Hz to 30 Hz.

juanmed commented 3 years ago

@karnikram Great news, thanks for sharing your config file! I will try it out.

amirmoeini commented 3 years ago

Hi @karnikram

Thanks for sharing the file. Can you please share some information on how the image rate of blackbird dataset can be changed? Thanks,

karnikram commented 3 years ago

Using the throttle node from the topic_tools package in ROS. If you're using a launch file, just add the following node element to it

<node name="throttle" pkg="topic_tools" type="throttle" args="messages /imu 100 /imu_throttled" />
GRF-Sunomikp31 commented 3 years ago

HI,i use the code but :catkin_make has a problem as follows: /home/shraouqiu/VIMO_WS/src/vimo/vimo_estimator/src/estimator.h:13:10: fatal error: blackbird/MotorRPM.h: No such file or directory

include <blackbird/MotorRPM.h>

amirmoeini commented 3 years ago

HI,i use the code but :catkin_make has a problem as follows: /home/shraouqiu/VIMO_WS/src/vimo/vimo_estimator/src/estimator.h:13:10: fatal error: blackbird/MotorRPM.h: No such file or directory

include <blackbird/MotorRPM.h>

Probably you cloned the master branch of the repository instead of the blackbird branch.

GRF-Sunomikp31 commented 3 years ago

HI,i use the code but :catkin_make has a problem as follows: /home/shraouqiu/VIMO_WS/src/vimo/vimo_estimator/src/estimator.h:13:10: fatal error: blackbird/MotorRPM.h: No such file or directory

include <blackbird/MotorRPM.h>

Probably you cloned the master branch of the repository instead of the blackbird branch. I follow the github： git clone https://github.com/uzh-rpg/vimo.git and git checkout blackbird

GRF-Sunomikp31 commented 3 years ago

HI,i use the code but :catkin_make has a problem as follows: /home/shraouqiu/VIMO_WS/src/vimo/vimo_estimator/src/estimator.h:13:10: fatal error: blackbird/MotorRPM.h: No such file or directory

include <blackbird/MotorRPM.h>

Probably you cloned the master branch of the repository instead of the blackbird branch.

by the way，do you know which dataset I should download？，.Is it this http://blackbird-dataset.mit.edu/BlackbirdDatasetData/star/yawConstant/maxSpeed1p0/rosbag.bag？But I can’t find the package star_maxSpeed1p0_images.bag

amirmoeini commented 3 years ago

HI,i use the code but :catkin_make has a problem as follows: /home/shraouqiu/VIMO_WS/src/vimo/vimo_estimator/src/estimator.h:13:10: fatal error: blackbird/MotorRPM.h: No such file or directory

include <blackbird/MotorRPM.h>

Probably you cloned the master branch of the repository instead of the blackbird branch. I follow the github： git clone https://github.com/uzh-rpg/vimo.git and git checkout blackbird

try this code git clone https://github.com/uzh-rpg/vimo.git -b blackbird instead of git clone https://github.com/uzh-rpg/vimo.git and see what happens.

amirmoeini commented 3 years ago

HI,i use the code but :catkin_make has a problem as follows: /home/shraouqiu/VIMO_WS/src/vimo/vimo_estimator/src/estimator.h:13:10: fatal error: blackbird/MotorRPM.h: No such file or directory

include <blackbird/MotorRPM.h>

Probably you cloned the master branch of the repository instead of the blackbird branch.

by the way，do you know which dataset I should download？，.Is it this http://blackbird-dataset.mit.edu/BlackbirdDatasetData/star/yawConstant/maxSpeed1p0/rosbag.bag？But I can’t find the package star_maxSpeed1p0_images.bag

The blackbird dataset are not as a bag file. you need first to install docker and use the code in https://github.com/mit-fast/Blackbird-Dataset/tree/f4667341c99869c7dc2d1176a8afc5d432dc403a to replay the datasets. There is problem in their instruction right now. however the following set of commands works for me (after installing docker)

HOST_STORAGE_DIR="$HOME/blackbirdDatasetData" BB_DATA_DIR="/root/blackbirdDatasetData"

docker pull winterg/flightgoggles_ros:ijrr

sudo docker run -it --rm --net=host\ -v "$HOST_STORAGE_DIR":"$BB_DATA_DIR" \ -p 10253-10263:10253-10263 \ winterg/flightgoggles_ros:ijrr \ /bin/bash

apt-get -y install nano

cd blackbirdDatasetTools/fileTreeUtilities/

nano sequenceDownloader.py

then change (you have to do this as the link is not updated in winterg/flightgoggles_ros:ijrr) serverAddress = "ijrr-blackbird-dataset.s3-accelerate.amazonaws.com/BlackbirdDatasetData" to serverAddress = "ijrr20-blackbird-dataset.s3-accelerate.amazonaws.com/BlackbirdDatasetData" and save

FLIGHT="star/yawForward/maxSpeed1p0" ENVIRONMENT="NYC_Subway_Station" $BB_TOOLS_DIR/fileTreeUtilities/sequenceDownloader.py --flight=$FLIGHT --environment=$ENVIRONMENT --datasetFolder=$BB_DATA_DIR

roslaunch blackbird playback_sequence.launch \ flight:=$FLIGHT \ environment:=$ENVIRONMENT \

datasetDir:=$BB_DATA_DIR

hope this also works for you.

fishpp-qq commented 3 years ago

HI,i use the code but :catkin_make has a problem as follows: /home/shraouqiu/VIMO_WS/src/vimo/vimo_estimator/src/estimator.h:13:10: fatal error: blackbird/MotorRPM.h: No such file or directory

include <blackbird/MotorRPM.h>

Probably you cloned the master branch of the repository instead of the blackbird branch.

by the way，do you know which dataset I should download？，.Is it this http://blackbird-dataset.mit.edu/BlackbirdDatasetData/star/yawConstant/maxSpeed1p0/rosbag.bag？But I can’t find the package star_maxSpeed1p0_images.bag

The blackbird dataset are not as a bag file. you need first to install docker and use the code in https://github.com/mit-fast/Blackbird-Dataset/tree/f4667341c99869c7dc2d1176a8afc5d432dc403a to replay the datasets. There is problem in their instruction right now. however the following set of commands works for me (after installing docker)

HOST_STORAGE_DIR="$HOME/blackbirdDatasetData" BB_DATA_DIR="/root/blackbirdDatasetData"

docker pull winterg/flightgoggles_ros:ijrr

sudo docker run -it --rm --net=host -v "$HOST_STORAGE_DIR":"$BB_DATA_DIR" -p 10253-10263:10253-10263 winterg/flightgoggles_ros:ijrr /bin/bash

apt-get -y install nano

cd blackbirdDatasetTools/fileTreeUtilities/

nano sequenceDownloader.py

then change (you have to do this as the link is not updated in winterg/flightgoggles_ros:ijrr) serverAddress = "ijrr-blackbird-dataset.s3-accelerate.amazonaws.com/BlackbirdDatasetData" to serverAddress = "ijrr20-blackbird-dataset.s3-accelerate.amazonaws.com/BlackbirdDatasetData" and save

FLIGHT="star/yawForward/maxSpeed1p0" ENVIRONMENT="NYC_Subway_Station" $BB_TOOLS_DIR/fileTreeUtilities/sequenceDownloader.py --flight=$FLIGHT --environment=$ENVIRONMENT --datasetFolder=$BB_DATA_DIR

roslaunch blackbird playback_sequence.launch flight:=$FLIGHT environment:=$ENVIRONMENT

datasetDir:=$BB_DATA_DIR

hope this also works for you.

HEELO！I wish to consult you on 2 questions. NO.1: When i execute "$BB_TOOLS_DIR/fileTreeUtilities/sequenceDownloader.py --flight=$FLIGHT --environment=$ENVIRONMENT --datasetFolder=$BB_DATA_DIR",there was a problem as follow.

Traceback (most recent call last): File "/usr/lib/python3.7/shutil.py", line 932, in _unpack_tarfile tarobj = tarfile.open(filename) File "/usr/lib/python3.7/tarfile.py", line 1578, in open raise ReadError("file could not be opened successfully") tarfile.ReadError: file could not be opened successfully

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/root/blackbirdDatasetTools/fileTreeUtilities/sequenceDownloader.py", line 125, in fire.Fire(downloadSequence) File "/usr/local/lib/python3.7/dist-packages/fire/core.py", line 138, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/usr/local/lib/python3.7/dist-packages/fire/core.py", line 468, in _Fire target=component.name) File "/usr/local/lib/python3.7/dist-packages/fire/core.py", line 672, in _CallAndUpdateTrace component = fn(*varargs, kwargs) File "/root/blackbirdDatasetTools/fileTreeUtilities/sequenceDownloader.py", line 118, in downloadSequence flight / environment / cam / timestampFile File "/root/blackbirdDatasetTools/fileTreeUtilities/sequenceDownloader.py", line 103, in download shutil.unpack_archive(dest, extract_dir=dest.with_name("images")) File "/usr/lib/python3.7/shutil.py", line 1002, in unpack_archive func(filename, extract_dir, kwargs) File "/usr/lib/python3.7/shutil.py", line 935, in _unpack_tarfile "%s is not a compressed or uncompressed tar file" % filename) shutil.ReadError: /root/blackbirdDatasetData/star/yawForward/maxSpeed1p0/NYC_Subway_Station/Camera_Left_Depth/lossless.tar is not a compressed or uncompressed tar file. NO.2: If I perform these operations, don't I need to download the dataset any more?

fishpp-qq commented 3 years ago

HI! I've used catkin_ make built VIMO successfully. However, blackbird dataset has only rosbag.bag but no star_maxSpeed1p0.bag and star_maxSpeed1p0_images.bag. What should i do at this step?(rosbag play YOUR_PATH_TO_DATASET/star_maxSpeed1p0.bag YOUR_PATH_TO_DATASET/star_maxSpeed1p0_images.bag )

aartykov commented 1 year ago

Hi @karnikram

were you able to run VINS-Mono the blackbird dataset using vimo? I am also seeing VINS-Mono fail on both star and picasso trajectories. The program outputs:

W0913 18:03:22.341883 22439 levenberg_marquardt_strategy.cc:114] Linear solver failure. Failed to compute a step: Eigen failure. Unable to perform dense Cholesky factorization.
[ WARN] [1599987802.399025387, 1525765667.055149141]: gyroscope bias initial calibration 0 0 0
[ INFO] [1599987802.400740441, 1525765667.055149141]: estimated scale: 0.000000
[ INFO] [1599987802.400880508, 1525765667.057669834]:  result g     0 0 0 0
[ INFO] [1599987802.400934201, 1525765667.057669834]: misalign visual structure with IMU

Hi, I met the same problem as yours. Could you run vimo without dynamics on blackbird?

Best regards