search

jihun1998 / AO

Official repository for CVPR 2024 paper: Weakly Supervised Point Cloud Semantic Segmentation via Artificial Oracle The code will be available soon.
11 stars 1 forks source link

Unable to repeat the results of the paper #2

Open huadonq opened 1 month ago

huadonq commented 1 month ago

I ran your code, and the miou of pp2s was 41, while the result in your paper was 50, which is quite a big difference.

jihun1998 commented 1 month ago

We confirm that the performance in our paper is reproducible. Can you provide the details about your trial, such as training log?

huadonq commented 1 month ago

Thanks for the reply, I attached my train.log file, in val only the highest miou is around 38, and the miou tested on area5 is 41. I also tried different setting, such as keeping the same parameters as in your supplementary material, but the results were almost the same. train.log

jihun1998 commented 1 month ago

Can you check quality of your pp2s-generated label? You can check by running "my_evaluate.py" in pointcept/engines folder with changing "gt_root" and "pred_root" variables.

huadonq commented 1 month ago

I get "sam label" with GT in "my_evaluate.py",follow result: image and get "weak label" with GT, follow: image

jihun1998 commented 1 month ago

Your generated sam label's quality is too low. We check that "my_run_sam_final.py" works fine. Please check if any other process has been done right.

huadonq commented 1 month ago

After careful investigation, I found that the following scene in the "used_imgs" you provided has no content, resulting in no "embedding" and "sam_label". Is this the problem that causes the low quality of sam_labels?

截屏2024-07-24 17 00 46

In the beginning, I can't run the code you gave directly, I need to change some configurations and other, etc. Please organize the code carefully, ensuring that others can run smoothly. Thanks again for your work!