search

isl-org / Open3D-ML

An extension of Open3D to address 3D Machine Learning tasks
Other
1.88k stars 321 forks source link

How to test the semantic segmentation results of TF version RandLA-Net in Open3D-ML? #573

Open whuhxb opened 1 year ago

whuhxb commented 1 year ago

Checklist

My Question

I have run the training code of RandLANet with TF version on Paris Lille dataset with the command: python scripts/run_pipeline.py tf -c ml3d/configs/randlanet_parislille3d.yml --dataset.dataset_path dataset/Paris-Lille-3D --pipeline SemanticSegmentation --dataset.use_cache True --num_workers 0 Which command to test the semantic segmentation results of RandLANet on Paris Lille dataset? I have tested with the following command, but get the wrong results. python scripts/run_pipeline.py tf -c ml3d/configs/randlanet_parislille3d.yml --split test --dataset.dataset_path dataset/Paris-Lille-3D --pipeline SemanticSegmentation --dataset.use_cache True --num_workers 0

What's the problem? Thanks in advance.

Started testing running inference Accuracy : [0.6033437, nan, nan, nan, nan, nan, nan, nan, nan, 0.6033437] IoU : [0.6033437, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.06703818888888889] running inference Accuracy : [0.563903, nan, nan, nan, nan, nan, nan, nan, nan, 0.563903] IoU : [0.563903, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0626558888888889] running inference Accuracy : [0.6278404, nan, nan, nan, nan, nan, nan, nan, nan, 0.6278404] IoU : [0.6278404, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.06976004444444445] Per class Accuracy : [nan, nan, nan, nan, nan, nan, nan, nan, nan] Per class IOUs : [nan, nan, nan, nan, nan, nan, nan, nan, nan] Overall Accuracy : nan Overall IOU : nan

v-parmar commented 1 year ago

Hey @whuhxb, The problem is that the testing results contain NaN values. The NaN values can be caused by various factors, including invalid calculations, such as dividing by zero or taking the square root of a negative number.

I suggest you some solutions, You can try it to resolve your issue:

  1. Make sure that the test dataset is loaded correctly by checking the path and the contents of the dataset.
  2. Check the model architecture and implementation for any bugs or errors that may cause invalid calculations.
  3. Check if any hyperparameters or configuration values are set to inappropriate values that may cause invalid calculations.
  4. Try using a different testing split or a smaller subset of the data to see if the problem is with the data or the model.
  5. Check for any preprocessing steps such as normalization or augmentation that may cause invalid values in the input data.
  6. Debug the code step by step to understand the cause of the NaN values and fix it accordingly.
dnnxin commented 6 months ago

Have you reslove this problem? I'm having the same issue