search

chrischoy / FCGF

Fully Convolutional Geometric Features: Fast and accurate 3D features for registration and correspondence.
MIT License
647 stars 112 forks source link

Evaluation Error for 3DMatch #17

Closed SwagJ closed 4 years ago

SwagJ commented 4 years ago

Dear Chris,

When I finished training and tried to evaluate the result following your instruction. It gives an error on mean of empty slice.

Full Error Message:

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 02/24 20:37:01 0.100000 0.050000 /disk_1/FCGF/scripts/benchmark_3dmatch.py:182: RuntimeWarning: Mean of empty slice. logging.info("average : %.4f +- %.4f" % (scene_r.mean(), scene_r.std())) /disk_1/anaconda3/envs/fcgf/lib/python3.7/site-packages/numpy/core/_methods.py:161: RuntimeWarning: invalid value encountered in double_scalars ret = ret.dtype.type(ret / rcount) /disk_1/anaconda3/envs/fcgf/lib/python3.7/site-packages/numpy/core/_methods.py:217: RuntimeWarning: Degrees of freedom <= 0 for slice keepdims=keepdims) /disk_1/anaconda3/envs/fcgf/lib/python3.7/site-packages/numpy/core/_methods.py:186: RuntimeWarning: invalid value encountered in true_divide arrmean, rcount, out=arrmean, casting='unsafe', subok=False) /disk_1/anaconda3/envs/fcgf/lib/python3.7/site-packages/numpy/core/_methods.py:209: RuntimeWarning: invalid value encountered in double_scalars ret = ret.dtype.type(ret / rcount) 02/24 20:37:01 average : nan +- nan ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Command input for running: python -m scripts.benchmark_3dmatch --source /disk_1/FCGF/3DMatch/threedmatch/ --target ./feature_tmp/ --voxel_size 0.025 --model /disk_1/FCGF/outputs/checkpoint.pth --do_generate --do_exp_feature --with_cuda

Do you have any idea on this error? Dataset is downloaded through command: ./scripts/download_datasets.sh /path/to/dataset/download/dir

And training is perfectly fine.

chrischoy commented 4 years ago

It seems that the test failed due to 0 / 0 error which means, it did not find any dataset and the result was 0; thus 0 / 0 --> NaN error.

I pushed a new commit with more assertions. Please make sure that you don't get any assertion errors during the evaluation.

SwagJ commented 4 years ago

Dear Chris,

If I understand it correctly, I need to download test data from this link https://github.com/andyzeng/3dmatch-toolbox#multi-frame-depth-tsdf-fusion and follow the instruction in Geometric Registration Benchmark section. And then, run your script.benchmark_3dmatch.py.

Is this correct?

chrischoy commented 4 years ago

The test set files are located at http://3dmatch.cs.princeton.edu under Geometric Registration Benchmark

SwagJ commented 4 years ago

Yes, it was my mistake. Thanks for reminding. However, in line 174 of benchmark_3dmatch.py:

traj = read_trajectory()

I think its input should be xxxx_gt.log instead of empty, right?

chrischoy commented 4 years ago

Ah thanks for pointing this out. I made this commit in a hurry and just fixed it after reading your comment on https://github.com/chrischoy/FCGF/commit/9d909cdfcb1556a919901e5bdb76b10f5b35375b

SwagJ commented 4 years ago

Thank you very much Chris. Everything works now. So I am gonna close this issue