9753 nan nan 0.00 nan nan 1698.001 16280.8

XuyangBai / D3Feat

[TensorFlow] Official implementation of CVPR'20 oral paper - D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features https://arxiv.org/abs/2003.03164

MIT License

261 stars 38 forks source link

9753 nan nan 0.00 nan nan 1698.001 16280.8 #23

Closed mtli77 closed 4 years ago

mtli77 commented 4 years ago

Hi @XuyangBai

I'm sorry to disturb you with the following questions. In the process of 3dmatch training, in 9753 step

Steps desc_loss det_loss train_accuracy d_pos d_neg time memory
9753 nan nan 0.00 nan nan 1698.001 16280.8.

What does this mean when the desc_loss and det_loss is nan, and the train_accuracy become 0.00, is it some kind of error in the training process?

Thank you very much and waiting for your reply!

XuyangBai commented 4 years ago

Hi @Violetit It seems very similar to the problem in here. You may check your TF and CUDA version first.

mtli77 commented 4 years ago

Hi @Violetit It seems very similar to the problem in here. You may check your TF and CUDA version first.

My CUDA version is 10.0 with the tensorflow-gpu=1.13.2