search

drprojects / superpoint_transformer

Official PyTorch implementation of Superpoint Transformer introduced in [ICCV'23] "Efficient 3D Semantic Segmentation with Superpoint Transformer" and SuperCluster introduced in [3DV'24 Oral] "Scalable 3D Panoptic Segmentation As Superpoint Graph Clustering"
MIT License
560 stars 72 forks source link

Issue with s3dis dataset- corrupt dataset #145

Closed samira-jpg closed 1 month ago

samira-jpg commented 1 month ago

Hello, I'm attempting to run SPT with s3dis dataset but when doing so I get the following error: "WARN - corrupted rgb data for file /content/superpoint_transformer/data/s3dis/raw/Area_5/office_19/Annotations/ceiling_1.txt". I'm aware there's a error in line 323474 of Area_5/office_19/Annotations/ceiling_1, but after I've fixed it but I keep getting errors at different lines in the file like in the picture, the error changes each time I run it. I'm aware a similar github issue was posted but it's unclear as to how that was resolved. any way I can resolve this?

Thank you Screenshot 2024-07-26 095247 Screenshot 2024-07-26 100333

drprojects commented 1 month ago

Hi @samira-jpg, thanks for your interest in the project ! I have not encountered this issue myself so I can't help much there.

I guess you have already seen issue #54. Have you already tried the two points I suggested there ?

If you ❤️ or simply use this project, don't forget to give the repository a ⭐, it means a lot to us !

samira-jpg commented 1 month ago

Hi, Thank you for the prompt response! Yes I've tried both of those I'm not using the Aligned version and I've tried redownloading the dataset. Is there any way to pinpoint what's causing this specifically?

Thank you

drprojects commented 1 month ago

Then I guess you should track which S3DIS files are problematic, isolate the problematic characters, and manually fix them. Since the S3DIS file reading is done in parallel to save up time, you might want to remove this parallelization for the sake of investigation. To this end, you may set read_s3dis_area(processes=1, ...) to only have one worker. Also, not that the error you get seems to suggest that the parser expected 6 characters but got 7 instead. Could this be that some of your lines have more separators (ie ' ') than other ? Could you maybe have some trailing spaces somewhere ?

PS: Don't forget to give the repository a ⭐, it means a lot to us !

samira-jpg commented 1 month ago

Thank you for the suggestion, I'm currently working on doing that. However I tried it with the mini s3dis (set mini=true) and somehow that managed to temporarily resolve the issue, although I'm not sure why, could you help me understand a bit more about the mini parameter as ideally I'd like to run it on the full dataset.

Also how can I visualise the final result?

drprojects commented 1 month ago

mini=True is not solving the problem. This is to reduce the size of the dataset for faster experimentation and debugging. So you no longer see the error because the problematic point clouds are simply not processed at all. The project's documentation and commented code should help you understand more what mini does.

For visualization, we provide notebooks in notebooks/ and documentation in docs/ as well as a general tutorial on Superpoint Transformer (see links in Readme).

This issue is related to S3DIS corrupt files. This is not something I can reproduce on my machines nor have much control over in the SPT codebase, so I am closing this issue. Please feel free to share your solution here if you happen to find one, in case someone else runs into the same problem.