search

ultralytics / yolov5

YOLOv5 🚀 in PyTorch > ONNX > CoreML > TFLite
https://docs.ultralytics.com
GNU Affero General Public License v3.0
50.9k stars 16.39k forks source link

Object detection knowledge transfer #11867

Closed MKlown closed 1 year ago

MKlown commented 1 year ago

Search before asking

Question

Hello,

I was able to generate a model from yolov5s with 3 new labels. It create a best.pt file that I can use with quite good results. However, it only contains my 3 new labels. I tried to have all exisiting labels before the new ones to the data/mymodel.yaml file. My new ones are respectively 80, 81 and 82. I tried to --freeze 0 and --freeze 24. I never been able to get good results. All my generated models only include my new labels (and data apparently). Could you please help me understanding what I am missing here?

Thanks a lot for any clue.

Additional

No response

glenn-jocher commented 1 year ago

@MKlown hi,

Thank you for reaching out and for providing the details of your query.

Based on your description, it seems like you are facing an issue with including existing labels in your model alongside the new labels you have added. To get a better understanding of the problem, could you please share the following information:

  1. The command or script you are using to train your model.
  2. The content of your data/mymodel.yaml file.
  3. Any relevant error messages or logs that you have encountered.

With this information, we will be able to assist you more effectively and provide you with the appropriate guidance.

Looking forward to your response.

Best regards.

MKlown commented 1 year ago

Hello,

Thanks again for the prompt answer. No error message during the training nor during the prediction. Just, it predicts only the masks or no mask (the new data). No more anything else from the used pretrained model during the tentative of transfer (diner table, bowl, fork, chair...). I tried without --freeze, with --freeze 0, with --freeze 24 Also by setting the \data\MaskNoMask.yaml file with only the new names at their respective position 80, 81, 82 and an nc = 3 (nc = 83 fail as expected in that precise case). Also by setting the new element alone with 0, 1, 2 for respectives numbers. All that to the result mentioned.

Here is the command: python .\yolov5\train.py --img 320 --cfg "yolov5s.yaml" --hyp "hyp.scratch-low.yaml" --batch 64 --epochs 100 --data "MaskNoMask.yaml" --weights "yolov5s.pt" --name "MaskNoMask" --cache --freeze 24

The dataset I am training is (converted from Pascal VOC to Yolov5 accordingly): https://www.kaggle.com/datasets/andrewmvd/face-mask-detection (I regenerate the set completely (unzip, convertion to Yolov5, renaming annotations folder to labels...) for every tries so, caches are not kept, data is "new". This is scripted for convenience). I do not use any other dataset since I would like to transfer so, not relearning what is existing in the small model yolov5s. The repartition into the 3 folders in 80/10/10%.

The MaskNoMask.yaml in \yolov5\data\

path: .\MaskNoMAsk\\ train: .\MaskNoMAsk\\images\train\ val: .\MaskNoMAsk\\images\val\ test: .\MaskNoMAsk\\images\test\

nc : 83

names: 0: person 1: bicycle 2: car 3: motorcycle 4: airplane 5: bus 6: train 7: truck 8: boat 9: traffic light 10: fire hydrant 11: stop sign 12: parking meter 13: bench 14: bird 15: cat 16: dog 17: horse 18: sheep 19: cow 20: elephant 21: bear 22: zebra 23: giraffe 24: backpack 25: umbrella 26: handbag 27: tie 28: suitcase 29: frisbee 30: skis 31: snowboard 32: sports ball 33: kite 34: baseball bat 35: baseball glove 36: skateboard 37: surfboard 38: tennis racket 39: bottle 40: wine glass 41: cup 42: fork 43: knife 44: spoon 45: bowl 46: banana 47: apple 48: sandwich 49: orange 50: broccoli 51: carrot 52: hot dog 53: pizza 54: donut 55: cake 56: chair 57: couch 58: potted plant 59: bed 60: dining table 61: toilet 62: tv 63: laptop 64: mouse 65: remote 66: keyboard 67: cell phone 68: microwave 69: oven 70: toaster 71: sink 72: refrigerator 73: book 74: clock 75: vase 76: scissors 77: teddy bear 78: hair drier 79: toothbrush 80: with_mask 81: without_mask 82: mask_weared_incorrect

Hope it helps.

glenn-jocher commented 1 year ago

@MKlown hello,

Thank you for providing the additional information. Based on your command and the contents of your data/mymodel.yaml file, it seems that you are on the right track. However, there are a few things you can try to ensure that your model predicts both the existing labels and the new ones.

First, make sure that you have a sufficient number of training examples for the existing labels in your dataset. If the dataset is dominated by examples of the new labels, the model may struggle to learn the existing labels effectively. Adding more training examples for the existing labels can help address this.

Second, you can experiment with adjusting the hyperparameters of your training process. You can try increasing the number of training epochs or adjusting other hyperparameters such as the learning rate to see if it improves the model's performance on the existing labels.

Lastly, you may consider using a larger pre-trained model, such as yolov5m or yolov5l, instead of yolov5s. These larger models tend to have better performance on a wider range of object classes.

I hope these suggestions help. If you have any further questions or need additional assistance, please let me know.

Best regards.

MKlown commented 1 year ago

Hello,

Not sure to get you here. "make sure that you have a sufficient number of training examples for the existing labels in your dataset" Since I am looking for a transfer, should I really have data to train the already existing learning? I understood transfer learning to not have to "relearn" the model on what it already knows. But adding new elements. In this particular case, if a pretrained model knows how to detect bikes, learning it to also detect cars should not implies to learn it again on detecting bikes. Just adding a car dataset should be enough. Am I wrong on this?

Thanks,

glenn-jocher commented 1 year ago

@MKlown hi,

Thank you for your question.

In transfer learning, the purpose is indeed to leverage the pre-trained model's knowledge on existing labels and enhance its ability to recognize new labels. You are correct that you do not need to retrain the model on the existing labels. However, it is still important to have a sufficient number of training examples for the existing labels in your dataset.

While the model does have previous knowledge of detecting certain objects like bikes, the additional training examples for the existing labels help fine-tune the model's understanding of these objects in the context of your specific dataset. This ensures that the model's performance on the existing labels is not compromised while it learns to recognize the new labels.

To summarize, you don't need to retrain the model completely on the existing labels, but including some training examples for these labels will help the model maintain its performance on them while learning the new labels.

Please let me know if you have any further questions or need additional clarification.

Thank you.

github-actions[bot] commented 1 year ago

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

MKlown commented 1 year ago

No worries. Completely understood you were not providing any support on reported problems so, another product has been choosed instead. Too much time loss already on this one.

github-actions[bot] commented 1 year ago

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

ilkkalyyvuo commented 1 year ago

@MKlown May I ask which product you decided to go with? I'm currently trying to achieve the same as your question here

glenn-jocher commented 1 year ago

@ilkkalyyvuo I'm sorry to hear that you've experienced difficulties with YOLOv5. If you have decided to switch to a different product, I hope it meets your requirements and provides a smoother experience for your project. If you have any further questions or need assistance with anything else, feel free to ask.