WongKinYiu / yolor

implementation of paper - You Only Learn One Representation: Unified Network for Multiple Tasks (https://arxiv.org/abs/2105.04206)
GNU General Public License v3.0
1.99k stars 518 forks source link

Question: how to add image class to model properly? #96

Open haxxolotl opened 3 years ago

haxxolotl commented 3 years ago

This is my steps so far:

  1. add new .cfg file, changing all classes=80 to classes=81, changing all filters=255 to filters=258
  2. add new yaml file, changing nc: 80 to nc:81, and adding a name to names: [] (i.e., empty shelf)
  3. add new .names file with the added name (i.e., empty shelf)
  4. download the pretrained yolor_w6.pt
  5. train the model with my custom data, using python train.py --batch-size 4 --img 1280 1280 --data my_yaml.yaml --cfg cfg/my_config.cfg --weights yolor_w6.pt --device 0 --name my_run --hyp hyp.scratch.1280.yaml --epochs 30
  6. inference with python detect.py --source inference/images/ --cfg cfg/my_config.cfg --weights runs/train/my_run/weights/last_029.pt --conf 0.2 --img-size 1280 --device 0 --names data/my_namesl.names

The output does indeed show my image classes being detected, but all the other ones are gone? I couldn't find any resources for proper steps for extending the model, so any help/correction with the above steps is appreciated

example non extended model inference: image

example attempted extended model inference, with desired "refrigerator" class missing: image

hiteshhedwig commented 3 years ago

Did you figure out anything?

haxxolotl commented 3 years ago

Did you figure out anything?

No, but I guess I have what I needed lol

WongKinYiu commented 3 years ago

because the shape of weights of the detection layers are different, they will be re-initialize by default.

in this case, to simplify the problem, i suggest you to append pseudo label of coco classes to your own label in your custom dataset.

(the better way is copy weights of original 80 classes to new weights, and do transfer/incremental learning on the new class.)