Custom dataset training clarification

[SkalskiP]

Hi team 👋!

First of all great project! I'm super excited to see that you used Detectron2 as framework of choice.

I'm trying to train my own model using custom dataset in COCO format. And for now I have 2 questions:

1. Do I need to provide annotations for all tasks? Thats how I understand those guidelines. I'm mostly interested in instance segmentation task:

coco/
  annotations/
    instances_{train,val}2017.json
    panoptic_{train,val}2017.json
    caption_{train,val}2017.json
    # evaluate on instance labels derived from panoptic annotations
    panoptic2instances_val2017.json
  {train,val}2017/
    # image files that are mentioned in the corresponding json
  panoptic_{train,val}2017/  # png annotations
  panoptic_semseg_{train,val}2017/  # generated by the script mentioned below

2. Do I need to train on 8x A100. I understand that you needed that much power when you trained from scratch, but if I use your checkpoint will 1x A100 be sufficient?

[praeclarumjj3]

Hi @SkalskiP, thanks for your interest in our work. Please find the answers to your questions below. I have tried to be as comprehensive as possible.

1. Annotations:

   • As we state in the paper, we only use panoptic annotations during our joint training process. Therefore, you need only the panoptic annotations for training the model.
   • However, we evaluate our model on the respective task annotations (task belongs to {panoptic, instance, semantic}). So, during data preparation, we prepare annotations for all tasks.
   • Custom Dataset Training: If you only want to train and test for the instance segmentation task, you can change the task sampling probabilities in the config files and set the default task as instance during testing. Note that we derive all labels from panoptic annotations in the dataset scripts, so prepare your files accordingly. https://github.com/SHI-Labs/OneFormer/blob/fea6b22e4b4cba00646abcc5872005b034dae83b/configs/coco/Base-COCO-UnifiedSegmentation.yaml#L46-L48 https://github.com/SHI-Labs/OneFormer/blob/fea6b22e4b4cba00646abcc5872005b034dae83b/configs/coco/oneformer_R50_bs16_50ep.yaml#L57

2. Training Resources:

   • I haven't tried training on a single GPU myself. I suggest training on at least 2 or 4 A100 GPUs because the hyper-parameters are set for multi-GPU training. Also, you might encounter OOM on a single GPU.

[praeclarumjj3]

Feel free to re-open if you face any issues.

[Ankitajadhav611]

Hey, I am trying to train a custom dataset for instance segmentation which is in the coco data format. I am able to train the model, it lacks bbox coordinate while inference, thus unable to infer the results.