What is the recommended way to train 2 new datasets with different classes via transfer learning in YOLOv5?

Nagidrop commented 2 years ago

Search before asking

[X] I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

This is my first question on GitHub and I really don't know if I ask it correctly, so I will try to include as many details as I could.

I'm currently working on a real time system to detect social distancing and face mask violations for my graduation project. Therefore, I have to include person and mask classes for training the system, namely 'person', 'no mask', 'correct mask', 'incorrect mask' (total of 4 classes). Currently, I have 3 different datasets, the first one only has objects labeled with 'person' class (dataset 1), the second one only has objects labeled with 'no mask', 'correct mask', 'incorrect mask' classes (dataset 2), and the third one is dataset 2 with my own augmentations (dataset 3).

The problem is, I cannot train the whole dataset with all 4 classes at once, as I need to train dataset 1 from pre-trained YOLOv5 weights, then use that weights file to train 2 times separately (not incremental but parallel from 1 checkpoint, 1st time with dataset 2, 2nd time with dataset 3) to do a comparison. I will use the same yaml file (with all 4 classes) for the training method described.

My question is, by not training the dataset as a whole (dataset 1 + 2 + 3), but only training dataset 1, then use the results to train dataset 2, 3 separately (using the same yaml file for all datasets with all 4 classes), does the latter training method affect the final results? The dataset 2 and 3 also have images containing people but do not have 'person' labels.

I also cannot have 2 separate weights file (one for dataset 1, the other for dataset 2, 3 to do comparison) as I need to detect all 4 classes by using only 1 detect code line (real time from camera stream), and as far as I know, YOLOv5 doesn't support inputting 2 weights into 1 --weights argument.

Your help is greatly appreciated and I'm really grateful for your support. Thank you all for reading my question, have a good day and wish you good health!

Additional

No response

github-actions[bot] commented 2 years ago

👋 Hello @Nagidrop, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email support@ultralytics.com.

Requirements

Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

Google Colab and Kaggle notebooks with free GPU:
Google Cloud Deep Learning VM. See GCP Quickstart Guide
Amazon Deep Learning AMI. See AWS Quickstart Guide
Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

glenn-jocher commented 2 years ago

@Nagidrop see https://community.ultralytics.com/t/how-to-combine-weights-to-detect-from-multiple-datasets/38/5 for a good discussion on training on multiple datasets.

Nagidrop commented 2 years ago

@glenn-jocher Many thanks for the link to the great discussion related to my problem. I'm really grateful for your prompt response. Now I can grasp the general idea and solution for my problem - basically I will have to combine them to 1 dataset and train, if not, I will have some other alternative solutions provided in the link.

However, I still wonder if the training method in my question still works for such cases, as there seems to be no reference in the given link. I will try to elaborate the question as below:

The training method I mentioned is simplified as: I have a yaml file containing 4 classes A, B, C, D for training. First, I train the model with a dataset containing only objects of A class (there are objects of B, C, D classes, but they are not labeled at all). Then, I train the same model with a dataset containing only objects of B, C, D classes (again, there are also objects of class A in this dataset, but they are not labeled at all). After training with these 2 datasets (using the given yaml having all A, B, C, D classes), will I achieve the same accuracy as if I train them as a combined dataset from the start, or the accuracy will be greatly affected?

I assume it falls into the latter case - decreased accuracy - as what I described is essentially partial labeling practices, but I'm not really sure, so could you please clarify it for me? Thank you in advance.

github-actions[bot] commented 2 years ago

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

Wiki – https://github.com/ultralytics/yolov5/wiki
Tutorials – https://docs.ultralytics.com/yolov5
Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

Ultralytics HUB – https://ultralytics.com/hub
Vision API – https://ultralytics.com/yolov5
About Us – https://ultralytics.com/about
Join Our Team – https://ultralytics.com/work
Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

urbansound8K commented 2 years ago

why did you close this issue? still not sloved!!

glenn-jocher commented 2 years ago

@urbansound8K our best guidance on this is in https://community.ultralytics.com/t/how-to-combine-weights-to-detect-from-multiple-datasets/38/5

Nagidrop commented 2 years ago

why did you close this issue? still not sloved!!

I didn't close it though, it was the bot which automatically closed it. However, I solved my own issue by training the dataset as a whole. I combined multiple datasets into one, using pseudo-labeling technique with pretrained weights to label the unlabeled parts of the datasets (label the mask classes for dataset with people only, and vice versa). Hopefully it will help you solve your issue.

JustBrandonLim commented 1 year ago

@glenn-jocher sorry for the old bump, I would like to ask if there's a way to also split the hyp for different datasets while combining them together to train a single model?

1 of my dataset (arrows) needs to have flipup and fliplr disabled, while the rest of my datasets should have it enabled.

glenn-jocher commented 11 months ago

@JustBrandonLim glad to hear you were able to resolve your issue! 🎉 Regarding your question, you can modify the --hyp file per dataset by creating a custom --hyp file for each dataset and loading it using the --hyp argument when training with that specific dataset. This way, you can specify different augmentation settings for each dataset. Let me know if you need any further assistance!

ultralytics / yolov5