Training for confirmation of S3DIS dataset results

[ba77-ku]

Have a nice day, congratulations on your work. I'm trying the architecture tutorial, but it doesn't go any further than the Ninja: not working to do error. Is it because I have only one GPU or do you have another solution suggestion?

[binjiechen]

Hi, the output is actually normal. No error occurred. Because the cpp extensions have been compiled, so ninja has no work to do, which is expected behavior. python train.py& (the &) makes the training script running in the background, and the output log (log and error info if any) will be written to output/log/01 by default. You can check things there. There will be new stuff after each epoch. disown detaches the job from the terminal, so closing the terminal won't kill the training process. Meanwhile, you can use nvidia-smi command to check GPU usage. For example, you can see that the training process is actually running, eating GPU memory and compute resources. To kill a running task, first find the task pid, like 6112 in your fig, or in the nvidia-smi output PID column. Then execute kill -9 pid I hope you find these helpful.

[ba77-ku]

Thank you for your return. How can we save the resulting prediction images as a point cloud? Additionally, I want to test it on my own data, but I don't fully understand the data format. What do I need to do to convert my own data into this format? It is in XYZ rgb format but I do not have a label, I want to test it directly, what can I do for this, thank you for your help.

[binjiechen]

If you mean to make predictions using the pretrained model, you first need to make sure the granularity/resolution/scale of your data is similar to S3DIS. Though I doubt it can achieve good results due to distribution shifts. If you mean to first train on your own dataset. Granularity is also important, because the default config is for S3DIS. Apart from that, depending on your data, other hyperparameters should also be adjusted for better results.

Then, to make a prediction, the data flow is: raw data -> preprocessed data -> test time data process (mainly subsample and knn search) -> model -> prediction. As detailed below.

S3DIS data is preprocessed by prepare_s3dis.py, which I believe is quite small and clear. You can check the data format here. Each room is preprocessed into xyz, rgb and label. xyz is made positive in line 79 of prepare_s3dis.py. rgb is 0-255.

In the dataset part, which loads preprocessed data and processes again and sends to the model, the getitem for train and test is different. You only need to adapt the get_test_item function in line 114 of s3dis.py for your needs. For example, you can just delete the label part and load your own data. Just make sure the data type of xyz and col is consistent. Besides that, "pick" decides which point in a grid to take. "loop" decides how many predictions are averaged in test time. These should be set according to your data. Though it's ok to leave it for now.

In test.py, you can delete the "y" (which is label) in line 32. "cum" is the accumulated logits, and you can save the final prediction after line 41 (which is the end of a loop). If you want, you can also save a room index (some numerical identifier) to pair with the coordinates afterwards. Simply add a room index in the end of get_test_item and receive it in line 32.

[ba77-ku]

Good day, but I still could not save the output 3d data. How should I edit the test .py to output the data? Good work.

[binjiechen]

Ok, so here are step-by-step instructions. Though not tested, I believe it should work.

1. Save paths of all processed test files to retain consistency. Suppose each file contains xyz, rgb as mentioned earlier. Edit paths and execute.

   import torch
   from pathlib import Path
   processed_test_data_path = Path("xxx")
   test_files_path_fn = Path("yyy/tfp.pt")
   paths = list(processed_test_data_path.glob(f'*.pt')) # or specify a pattern like test*.pt
   torch.save(paths, test_files_path_fn)

   Then replace line 28 of s3dis.py with

   self.paths = torch.load("yyy/tfp.pt")

   Edit the path to match with test_files_path_fn. You can use an if test: clause if you like.

2. Now about get_test_item func. First replace line 119 with

   xyz, col = self.datas[idx]

   Then replace line 140 with

   return xyz, feature, indices, full_nn, idx

   Lastly remove line 122.

3. Finally the test.py. First line 32 to

   for xyz, feature, indices, nn, idx in testdlr:

   Then replace line 42, 43 with

   p = cum.argmx(dim=1)
   torch.save(p, pred_save_folder / paths[idx].name)

   Lastly load paths and set pred_save_folder anywhere before the test loop.

   paths = torch.load("yyy/tfp.pt")
   pred_save_folder = Path("zzz")   # create the folder if it does not exist

   All done now. Execute.

4. Load predictions.

   import torch
   from pathlib import Path
   paths = torch.load("yyy/tfp.pt")
   pred_save_folder = Path("zzz")
   # Read point cloud 0
   idx = 0
   xyz, col = torch.load(paths[idx])
   pred = torch.load(pred_save_folder / paths[idx].name)