Closed Hanziscool closed 2 years ago
wenbowen123
commented
2 years ago Hi, can you download the latest code and change to =1? https://github.com/wenbowen123/BundleTrack/blob/748d5693a5b4ac29ae65eff84c4b128816e29dab/CMakeLists.txt#L19
Then run again rm -rf build && mkdir build && cd build && cmake .. && make
This will disable opencv's GPU running with a slower running speed. See if this fix your problem.
Hanziscool
commented
2 years ago fixed, can run predictions & evaluations on ycbineoat now thanks a lot!
wenbowen123
commented
2 years ago Following https://github.com/wenbowen123/BundleTrack/issues/9#issuecomment-1001982186
Did you re-compile by rm -rf build && mkdir build && cd build && cmake .. && make?
You mentioned you didn't see anything in the debug_dir. Can you first make sure you have that folder created?
Hanziscool
commented
2 years ago yes i have re-compiled and run the code previously and it is still showing
frame %s and %s findNN too few match, %s status marked as FAIL
@wenbowen123 i have attached the outputs in the debug_dir. https://drive.google.com/drive/folders/1pooX1QakKrywwz8IRDwvGkrnFtaQi-bD?usp=sharing
I found that it can only produce yellow keypoints for the initial object pose, but able to track the segmentation shown in the folder color_raw
Hanziscool
commented
2 years ago 
@wenbowen123 Here it is unable to produce the right keypoints when the object is moved away from the starting pose.
wenbowen123
commented
2 years ago Did you change the LOG level to 3 or higher? Your logged data seems incomplete.
Also make sure the detected image is saved and check how many keypoints are detected around here: https://github.com/wenbowen123/BundleTrack/blob/4c4a5c0a70ac6808d92fbca615c5d2c16370aaee/src/FeatureManager.cpp#L130
Hanziscool
commented
2 years ago yes @wenbowen123 i tried with LOG 3 and i am not seeing any more logged data. The detected image is not saved either. Does it have to do with the commented detector in FeatureManager.cpp from the latest commit?
wenbowen123
commented
2 years ago Can you keep LOG=3 and copy the complete (or as much as you can) run output from your terminal to here?
Hanziscool
commented
2 years ago output from run_ycbineoat.py
mask dir is /home/jrojas/BundleTrack/scripts/../masks/ycbineoat/tomato_soup_can_yalehand0/masks and code dir is /home/jrojas/BundleTrack/scripts
/home/jrojas/BundleTrack/scripts/../build/bundle_track_ycbineoat /tmp/config_tomato_soup_can_yalehand0.yml
cam K=
319.582 0 320.215
0 417.119 244.349
0 0 1
ob_in_cam0
0.0376972 -0.0101344 -0.999238 0.0716899
0.994209 -0.100319 0.038525 0.122491
-0.100633 -0.994904 0.00629392 0.770057
0 0 0 1
data has 1308 images Connected to port 5555 color file: /home/jrojas/YCBInEOAT/tomato_soup_can_yalehand0/rgb/1582751106936950586.png
New frame 1582751106936950586 zmq start waiting for reply zmq got reply color file: /home/jrojas/YCBInEOAT/tomato_soup_can_yalehand0/rgb/1582751106969170614.png
New frame 1582751106969170614 zmq start waiting for reply zmq got reply finding corres between 1582751106969170614(id=1) and 1582751106936950586(id=0) frame 1582751106969170614 and 1582751106936950586 findNN too few match, 1582751106969170614 status marked as FAILcolor file: /home/jrojas/YCBInEOAT/tomato_soup_can_yalehand0/rgb/1582751107228941885.png
New frame 1582751107228941885 zmq start waiting for reply zmq got reply finding corres between 1582751107228941885(id=1) and 1582751106936950586(id=0) frame 1582751107228941885 and 1582751106936950586 findNN too few match, 1582751107228941885 status marked as FAILcolor file: /home/jrojas/YCBInEOAT/tomato_soup_can_yalehand0/rgb/1582751107296592676.png
output from lfnet run_server.py
/opt/conda/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
return f(*args, **kwds)
Loading from /home/jrojas/BundleTrack/lf-net-release/release/models/indoor/config.pkl
---------------------- OPTIONS ----------------------
activ_fn leaky_relu
aug_max_degree 180
aug_max_scale 1.4142135623730951
batch_size 6
clear_logs False
com_strength 3.0
conv_ksize 5
crop_radius 16
data_raw_size 362
data_size 256
dataset scannet
depth_thresh 1.0
desc_activ_fn relu
desc_conv_ksize 3
desc_dim 256
desc_inputs photos
desc_leaky_alpha 0.2
desc_loss triplet
desc_margin 1.0
desc_net_channel 64
desc_net_depth 3
desc_norm l2norm
desc_perform_bn True
desc_train_delay 0
descriptor simple_desc
det_loss l2loss
detector mso_resnet_detector
do_softmax_kp_refine True
hard_geom_thresh False
hm_ksize 15
hm_sigma 0.5
hpatches_dir /scratch/trulls/yuki.ono/datasets/hpatches
init_num_mine 64
input_inst_norm True
kp_com_strength 1.0
kp_loc_size 9
leaky_alpha 0.2
log_dir /scratch/trulls/yuki.ono/results/deep_det/desc/180705-scannet-ori-sv-3d/adam-lr-1e-3-False/mso_resnet_detector/scannet-15/aug-rTrue-180-sTrue/ori-True-5-scl-5-0.7-1.4/desc-photos-D256-topk-512/mine-rand_hard_sch-64-5-0.9/innorm-True/wdet-0.01/ori-l2loss-w-0.1/scl-0.1/nms3d-True/try-1
lr 0.001
lr_decay False
match_reproj_thresh 5
max_itr 50000
max_seq_length 2000
min_num_pickup 5
mining_type rand_hard_sch
net_block 3
net_channel 16
net_max_scale 1.4142135623730951
net_min_scale 0.7071067811865475
net_num_scales 5
nms_ksize 5
nms_thresh 0.0
num_threads 16
optim_method adam
ori_ksize 5
ori_loss l2loss
ori_weight 0.1
patch_size 32
perform_bn True
pickup_delay 0.9
pretrain_dir
random_offset False
rot_aug True
scale_aug True
scale_com_strength 100.0
scale_weight 0.1
scannet_dir /scratch/trulls/yuki.ono/datasets/scannet/dataset
scenenet_dir /scratch/trulls/yuki.ono/datasets/scenenet
score_com_strength 100.0
sfm_dpt_dir /scratch/trulls/yuki.ono/datasets/colmap/dataset2/vis-0.4
sfm_img_dir /scratch/trulls/yuki.ono/datasets/colmap/colmap
sfm_mode nips
sfm_seq sacre_coeur
sfm_train_seq train.txt
sfm_valid_seq valid.txt
show_histogram False
sm_ksize 15
soft_kpts True
soft_scale True
top_k 512
train_main_seq 0
train_num_traj 100
train_ori True
train_pair_offset 15
train_same_time True
use_nms3d True
valid_num_traj 10
valid_pair_offset 15
webcam_dir /scratch/trulls/yuki.ono/datasets/WebcamRelease
weight_det_loss 0.01Act-Fn: <function get_activation_fn.
FLAT (?, 4096) Feat-Norm: L2-NORM FEAT (?, 256) 2022-01-05 07:11:29.448931: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA 2022-01-05 07:11:29.597981: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties: name: NVIDIA RTX A6000 major: 8 minor: 6 memoryClockRate(GHz): 1.8 pciBusID: 0000:9e:00.0 totalMemory: 47.54GiB freeMemory: 23.00GiB 2022-01-05 07:11:29.598059: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: NVIDIA RTX A6000, pci bus id: 0000:9e:00.0, compute capability: 8.6) Load trained models... Checkpoint models-latest-42000 port tcp://:5555 lfnet listending to tcp://:5555 W=400, H=400 lfnet listending to tcp://*:5555 W=400, H=400
@wenbowen123 I noticed you disabled the detector in Featuremanager.cpp in the latest commit, does this also remove the key point detection?
wenbowen123
commented
2 years ago Can you add a line _fm->vizKeyPoints(frame) at https://github.com/wenbowen123/BundleTrack/blob/4c4a5c0a70ac6808d92fbca615c5d2c16370aaee/src/Bundler.cpp#L118
and check the output in the debug_dir?
kausiksivakumar
commented
2 years ago Following #9 (comment)
Did you re-compile by
rm -rf build && mkdir build && cd build && cmake .. && make?You mentioned you didn't see anything in the debug_dir. Can you first make sure you have that folder created?
I changed the things as you mentioned but I could not find visualizations for NOCS. The debug directory contains /tmp/BundleTrack/nocs/can_arizona_tea_norm_1 . These contain color_viz (which is empty) and poses that txt files each holding homogeneous transformation matrix. Could you help with the visualizations. Thanks
Everything is currenly running inside docker. Packages from pip list shows these which are aligned with the one in lf-net-release-env:
Top bash is running python run_server.py, running properly.
Bottom bash is running ython scripts/run_ycbineoat.py --data_dir /home/jrojas/YCBInEOAT/tomato_soup_can_yalehand0 --port 5555 --model_name 005_tomato_soup_can --model_dir /home/jrojas/YCBInEOAT/YCB_Video_Dataset/models/005_tomato_soup_can/textured_simple.obj /home/jrojas/BundleTrack/scripts/../build/bundle_track_ycbineoat /tmp/config_YCBInEOAT.yml
I downloaded https://archive.cs.rutgers.edu/archive/a/2020/pracsys/Bowen/iros2020/YCBInEOAT/https://archive.cs.rutgers.edu/archive/a/2020/pracsys/Bowen/iros2020/YCBInEOAT/ and https://drive.google.com/file/d/1gmcDD-5bkJfcMKLZb3zGgH_HUFbulQWu/view . I put them in /home/usrname/YCBInEOAT . However i noticed that when i run predictions, its looking for a missing masks folder, so i downloaded https://archive.cs.rutgers.edu/archive/a/2021/pracsys/2021_iros_bundletrack/masks.tar.gz and put it inside /home/usrname/YCBInEOAT/tomato_soup_can_yalehand0/