search

shipra25jain / ESSNet

Embedding-based Scalable Segmentation Network
MIT License
28 stars 6 forks source link

Loading coco_lvis images and segmentation masks #1

Closed JessLittle-art closed 3 years ago

JessLittle-art commented 3 years ago

@shipra25jain , thanks for sharing your great work. I want to train your code on coco_lvis dataset. I am confused of the way that you load the images and their segmentation masks. I wonder what are coco2lvis.pkl, color_map_lvis.pkl, img2info.pkl, img2info_val.pkl and where you got them.

shipra25jain commented 3 years ago

Hi @JessLittle-art As mentioned in the paper, our COCO_LVIS dataset is built by merging semantic segmentation annotations of stuff classes from COCO-Stuff dataset and only semantic labels (for object classes) from instance segmentation annotations from LVIS dataset for COCO2017 images. So, we merge the stuff classes from COCO and object classes from LVIS to get a list of 1200 something classes. However, there are few classes which are common in both the dataset, so we take the union of those classes. And to take care of this, we have a mapping called coco2lvis.pkl which says which class number in coco matches to which class number in lvis.

color_map.pkl is just color palette for 1284 classes to visualize the segmentation mask. img2info_val.pkl and img2info.pkl are just dictionaries for training and validation split with {image_id :{'path':xyz, 'height':xyz,'width':xyz}. Lastly, to be able to build the dataset, I would suggest you to follow these steps:

  1. create folder 'lvisdataset/lvis_v1_train.json' (this file you will find on lvis dataset website)
  2. create folder 'cocodataset/annotations/stuff_train2017.json' ( you will find this json on coco dataset website)
  3. download COCO 2017 images (available on both coco and lvis websites)
  4. run the scripts in generateMasks.py and generateValMasks.py to generate semantic segmentation masks for COCO2017 images with 1284 classes

Let me know if you have more questions.

Shipra