search

mseg-dataset / mseg-api

An Official Repo of CVPR '20 "MSeg: A Composite Dataset for Multi-Domain Segmentation"
Other
249 stars 45 forks source link

MSEG CamVid test set: is listed but not remapped? #17

Closed tmensink closed 4 years ago

tmensink commented 4 years ago

I noticed that for camvid the test set seems to be listed (ie the TSV file camvid-11/list/test.txt exists).

However after downloading and remapping, the semseg11 segmentation mask is missing (eg semseg11/0001TP_008550_L.png is missing), while the original 32 class segmentation mask does exists (Labels32-RGB/0001TP_008550_L.png).

How can I remap the test set as well?

johnwlambert commented 4 years ago

Hi Thomas (@tmensink ), thanks for your interest in our work. We only remap the 'train' and 'val' splits by default (https://github.com/mseg-dataset/mseg-api/blob/master/mseg/label_preparation/remap_dataset.py#L78). You can add 'test' to that line to remap it.

Since we were repeating many experiments and evaluation, we wanted to make sure we didn't "cheat" by evaluating on the test set multiple times.

johnwlambert commented 4 years ago

Closing for now, please feel free to re-open if you have further questions.