Closed jsherrah closed 11 years ago
Here are the things we need to know for MSRC experiments in the literature:
I think Shotton's "TextonBoost for Image Understanding" ICJV 2007 paper introduced the MSRC dataset to the community. Shotton published the split used in there experiments: http://jamie.shotton.org/work/data/TextonBoostSplits.zip
1) Klucker et al [Super-Pixel Class Segmentation in Large-Scale Aerial Imagery, 2010] et al state: The dataset includes 532 images with 21 object classes. As suggested in [Shotton et al TextonBoost, 2007], 276 images are used for training and the remaining 256 for testing.
2) Gould et al [Multi-Class Segmentation with Relative Location Prior, 2008] states: Following the protocol of previous works on this database [22, 24, 31], we ignore void pixels during both training and evaluation. We over-segment our images using the method of Ren and Malik [20], tuned to give approximately 400 superpixels per image. We randomly divide the images into a training set with 276 images and a testing set with 315 images, allowing us to be directly comparable to the results of Shotton et al. [24 - Shotton TextonBoost, 2007].
I'd say it is safe to follow in their footsteps and use the "protocol" defined in Shotton's 2007 TextonBoost paper for dealing with the MSRC Dataset.
TextonBoost paper "protocol":
1) 591 images
2) 21 object classes: building, grass, tree, cow, sheep, sky, airplane, water, face, car, bicycle, flower, sign, bird, book, chair, road, cat, dog, body, and boat.
3) Split ratio: Approx. 45% training, 10% validation and 45% test sets, approx. proportional contributions from each class.
See http://jamie.shotton.org/work/data/TextonBoostSplits.zip
What are the excluded classes? Which images do they appear in?
Is the class exclusion automatic, in that if we use the given images, those classes won't appear? If not, then we have two choices:
or are people testing on the training data? What are the common hacks to make results comparable. How is accuracy measured