About segmentation and evaluation method in the test sequences of semanticKITTI dataset

[AkinoriKotani]

Hi, Thanks for your great opening source of this project!

I have 3 questions about the segmentation and evaluation method in the test sequences.

1.) About the predicted value is not saved when I run Eval-Script Now I can run Training-Script and Eval-Script using sequences 00-10, respectively. First, when I ran the Training-Script, directories such as predictions were created in thelogs/2020-11-17-23: 08 salsanext-cpdirectory. However, when the Eval-Script was executed, the pred directory for storing the value of prediction was created, but the contents of pred/sequences/<sequence No.>/predictions were all empty. The run-time command is $ ./eval.sh -d ~/Dataset/SemanticKITTI/dataset -p pred -m logs2/logs/2020-11-17-23: 08 salsanext-cp -s valid -c 30 -n salsanext. I don't know what the problem is, so I would appreciate it if you could tell me.

2.) About labels of test sequences. Next, I would like to run test and evaluate using the test sequences, but I do not know how to carry out the experiment because there is no labels. Is it generated as a predicted value when I run some program? or if not necessary? I would appreciate it if you could tell me how to run these scripts.

3.) How to use visualize.py First, is there a way to save the results? And then also, like Question 2, I don't know how to run it in an unlabeled test sequences. Is the labels still something that can be generated by running some program in SalsaNext, or is the option instead of the label the value of predictions that should be generated by Eval-Script with -p? It seems that visualize.py is not executed using the execution result of SalsaNext, so I would appreciate it if you could tell me how to use visualize.py.

I am very sorry for the amateur questions. thank you.

[TiagoCortinhal]

Hello @AkinoriKotani

1. Did you have the predictions for sequence 8? (the validation set)

2. The test sequence labels are not openly available. You will need to go to the semantickitti codalab to run the evaluation there.

3. You can tweak the script to save the visualisation's output if you'd like. To run it you'd need to use -d (for the dataset) -c (for the config yml file) -s (for the desired sequence) and yes -p for your predictions. To save it you can check the class LaserScanVis (under train.common) .

[AkinoriKotani]

Thank you for teaching me while you are busy. I immediately tried what you taught me.

1.) About the predicted value Initially, the prediction labels for all sequences weren't saved, but when I ran eval.sh again, the labels were saved. The cause is unknown, but it has been resolved for the time being. Thank you so much.

2.) About labels of test sequences I understand. I will go to the semantickitti codalab to run the Eval-Script there.

3.) About visualize.py Thanks for teaching. First, when I specified the pred folder generated by Eval-Script with -p, the results was output safely. Next, regarding the saving method, I reread laserscanvis.py, but I could not understand where the code to save was written, so could you tell me what line it is written.

Thank you.

[TiagoCortinhal]

3. The code is not actually there. You can, if you want adapt it to save images. For example you can save the projected segmentation by plt.imsave('filename.png',self.scan.proj_sem_color[..., ::-1])

[AkinoriKotani]

Thank you for your teaching.

When I added plt.imsave('filename.png',self.scan.proj_sem_color[..., ::-1]) ↓ plt.imsave('sem_img' + str(self.offset) + '.png', self.scan.proj_sem_color[..., ::-1]) shutil.move('sem_ing' + str(self.offset) + '.png', 'dir_path to save') to the appropriate place below laserscanvis.py, I was able to save it as elongated 2D images for each offset. (line:183, 187, and 191) I think there is a better way to write it, but this was the limit for me.

In any case, I was able to solved it. Thank you very much for your kind guidance during your busy schedule. I pray for your continued success!