search

OpenSun3D / cvpr24-challenge

OpenSUN3D Workshop Challenge - CVPR '24
16 stars 0 forks source link

Eval Script #2

Open parodiniccolo opened 1 month ago

parodiniccolo commented 1 month ago

Hi all, It seems that the demo_eval.py script doesn't like loading scene files with a single mask txt file :)

For example, if 42445935.txt only contains:

predicted_masks/42445935_000.txt 1.0

the assertion error at line 22 gets raised.

It's a simple fix, but I thought it was good to point it out if you use the same script for the final evaluation on the test set :)

aycatakmaz commented 1 month ago

Hi Niccolò,

Thanks a lot for letting us know and for your participation in our workshop! As we have two tracks with two respective evaluation scripts, I wanted to understand whether you encountered the error using https://github.com/OpenSun3D/cvpr24-challenge/blob/main/challenge_track_1/demo_eval.py or https://github.com/OpenSun3D/cvpr24-challenge/blob/main/challenge_track_2/benchmark_eval/demo_eval.py file. These scripts are almost the same except for a few details in the submission reading part. Since the eval script with the assertion error at line 22 is the one for the track 2, I wanted to make sure you are using the correct evaluation script in case you are working on track 1!

Best, Ayca

parodiniccolo commented 1 month ago

Hi Ayca,

Yes sorry! I've mentioned line 22 but actually is line 17 on the first track, I'm using the correct one, just mentioned you the "wrong" one, but I guess the problem will stand in both the eval scripts :)

BR