HimangiM / Just-Go-with-the-Flow-Self-Supervised-Scene-Flow-Estimation

Self-supervised method for scene-flow estimation of LiDAR point clouds. Method is trained and tested on the nuScenes and KITTI datasets in TensorFlow. (CVPR 2020)
BSD 3-Clause "New" or "Revised" License
60 stars 5 forks source link

Not being able to reproduce the results in the paper #35

Open ry4nzhu opened 2 years ago

ry4nzhu commented 2 years ago

Hi, thanks for sharing the code! It is a very cool project. I tried to setup the project and reproduce the results from the paper using the pre-trained model provided. However, I'm getting very unreasonable average EPE metric when evaluating on the KITTI dataset (as large as 1081427622). Below are the full outputs when I tried to run:

~/cooperative-perception/just-go-flow$ bash src/commands/command_evaluate_kitti.sh 
/home/ryanzhu/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/ryanzhu/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/ryanzhu/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/ryanzhu/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/ryanzhu/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/ryanzhu/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:532: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
Namespace(batch_size=8, data='data_preprocessing/kitti_self_supervised_flow', decay_rate=0.7, decay_step=200000, flow='default', gpu=0, kitti_dataset='data_preprocessing/kitti_self_supervised_flow', knn=False, layer='pointnet', model='model_concat_upsa', model_path='log_train_pretrained/model.ckpt', num_frames=2, num_point=2048, radius=5.0)
--- Get model and loss
check for seg fault
WARNING:tensorflow:From /home/shawn/cooperative-perception/just-go-flow/src/utils/pointnet_util.py:183: calling reduce_max (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
check for seg fault
check for seg fault
check for seg fault
Radius here: 5.0
KNN False
flow module default
knn
check for seg fault
check for seg fault
WARNING:tensorflow:From /home/shawn/cooperative-perception/just-go-flow/src/utils/pointnet_util.py:286: calling reduce_sum (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From /home/ryanzhu/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py:497: calling conv1d (from tensorflow.python.ops.nn_ops) with data_format=NHWC is deprecated and will be removed in a future version.
Instructions for updating:
`NHWC` for data_format is deprecated, use `NWC` instead
2022-06-27 18:29:20.808658: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2022-06-27 18:29:20.875689: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:897] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27 18:29:20.876026: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1392] Found device 0 with properties: 
name: NVIDIA GeForce RTX 3080 major: 8 minor: 6 memoryClockRate(GHz): 1.71
pciBusID: 0000:01:00.0
totalMemory: 9.75GiB freeMemory: 8.80GiB
2022-06-27 18:29:20.876038: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1471] Adding visible gpu devices: 0
2022-06-27 18:31:37.852776: I tensorflow/core/common_runtime/gpu/gpu_device.cc:952] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-06-27 18:31:37.852794: I tensorflow/core/common_runtime/gpu/gpu_device.cc:958]      0 
2022-06-27 18:31:37.852799: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0:   N 
2022-06-27 18:31:37.852874: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1084] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8490 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3080, pci bus id: 0000:01:00.0, compute capability: 8.6)
12250 12250
16557 16557
18910 18910
21888 21888
38359 38359
8594 8594
29002 29002
7928 7928
18281 18281
17872 17872
13034 13034
9272 9272
30121 30121
14296 14296
15659 15659
32019 32019
27487 27487
28991 28991
22131 22131
14037 14037
15777 15777
22336 22336
11374 11374
14142 14142
33599 33599
27657 27657
9850 9850
12332 12332
38519 38519
41886 41886
13028 13028
24917 24917
20459 20459
11022 11022
30124 30124
34816 34816
38604 38604
29192 29192
18416 18416
49152 49152
22012 22012
13785 13785
47287 47287
33916 33916
27527 27527
20393 20393
11501 11501
22157 22157
14149 14149
20655 20655
(50,) (50,) (50,)
Num batches 1064960.0 Average EPE 1081427622.2777448
eval mean EPE 3D: 1081427622.277745

Do you know what could cause such unreasonable behavior? Thanks.

derkaczda commented 2 years ago

Hello @ry4nzhu, any progress on this one? I have the same problem, also if I visualize the point clouds and compare them to the ground truth flow it is for most points not even close to the ground truth, did you maybe experience similar issues?

ry4nzhu commented 2 years ago

Which type of GPU you're using? I replicate the same setup for a machine with RTX 2060 and it worked. The same setup dose not work on a machine RTX 3080. I'm actually not sure what causes this.

derkaczda commented 2 years ago

Now that you mention it, I had a similar experience: Could reproduce it on a Quadro RTX 5000 but not on a RTX 3070

ddfine commented 1 year ago

Hi, thanks for sharing the code! It is a very cool project. I tried to setup the project and reproduce the results from the paper using the pre-trained model provided. However, I'm getting very unreasonable average EPE metric when evaluating on the KITTI dataset (as large as 1081427622). Below are the full outputs when I tried to run:

~/cooperative-perception/just-go-flow$ bash src/commands/command_evaluate_kitti.sh 
/home/ryanzhu/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/ryanzhu/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/ryanzhu/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/ryanzhu/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/ryanzhu/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/ryanzhu/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:532: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
Namespace(batch_size=8, data='data_preprocessing/kitti_self_supervised_flow', decay_rate=0.7, decay_step=200000, flow='default', gpu=0, kitti_dataset='data_preprocessing/kitti_self_supervised_flow', knn=False, layer='pointnet', model='model_concat_upsa', model_path='log_train_pretrained/model.ckpt', num_frames=2, num_point=2048, radius=5.0)
--- Get model and loss
check for seg fault
WARNING:tensorflow:From /home/shawn/cooperative-perception/just-go-flow/src/utils/pointnet_util.py:183: calling reduce_max (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
check for seg fault
check for seg fault
check for seg fault
Radius here: 5.0
KNN False
flow module default
knn
check for seg fault
check for seg fault
WARNING:tensorflow:From /home/shawn/cooperative-perception/just-go-flow/src/utils/pointnet_util.py:286: calling reduce_sum (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From /home/ryanzhu/anaconda3/envs/flow/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py:497: calling conv1d (from tensorflow.python.ops.nn_ops) with data_format=NHWC is deprecated and will be removed in a future version.
Instructions for updating:
`NHWC` for data_format is deprecated, use `NWC` instead
2022-06-27 18:29:20.808658: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2022-06-27 18:29:20.875689: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:897] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2022-06-27 18:29:20.876026: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1392] Found device 0 with properties: 
name: NVIDIA GeForce RTX 3080 major: 8 minor: 6 memoryClockRate(GHz): 1.71
pciBusID: 0000:01:00.0
totalMemory: 9.75GiB freeMemory: 8.80GiB
2022-06-27 18:29:20.876038: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1471] Adding visible gpu devices: 0
2022-06-27 18:31:37.852776: I tensorflow/core/common_runtime/gpu/gpu_device.cc:952] Device interconnect StreamExecutor with strength 1 edge matrix:
2022-06-27 18:31:37.852794: I tensorflow/core/common_runtime/gpu/gpu_device.cc:958]      0 
2022-06-27 18:31:37.852799: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0:   N 
2022-06-27 18:31:37.852874: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1084] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 8490 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 3080, pci bus id: 0000:01:00.0, compute capability: 8.6)
12250 12250
16557 16557
18910 18910
21888 21888
38359 38359
8594 8594
29002 29002
7928 7928
18281 18281
17872 17872
13034 13034
9272 9272
30121 30121
14296 14296
15659 15659
32019 32019
27487 27487
28991 28991
22131 22131
14037 14037
15777 15777
22336 22336
11374 11374
14142 14142
33599 33599
27657 27657
9850 9850
12332 12332
38519 38519
41886 41886
13028 13028
24917 24917
20459 20459
11022 11022
30124 30124
34816 34816
38604 38604
29192 29192
18416 18416
49152 49152
22012 22012
13785 13785
47287 47287
33916 33916
27527 27527
20393 20393
11501 11501
22157 22157
14149 14149
20655 20655
(50,) (50,) (50,)
Num batches 1064960.0 Average EPE 1081427622.2777448
eval mean EPE 3D: 1081427622.277745

Do you know what could cause such unreasonable behavior? Thanks.

Hi, have you replicated the code of "Just Go with the Flow: Self-Supervised Scene Flow Estimation "? I have some questions about this. First,how to visualize the result like the paper? Second,is the accuracy problem of your reproduction results solved?