Tips for fine-tune

[Fiorentinar]

Thank you very much for your work!

I currently have a dataset with the following characteristics: high resolution(1920*1080), very limited data amount(~500 frames), and specific category names( just like 'bottom of trunk' or 'top of the traffic sign'). Specifically, these are all small objects for detection. It can be said that the dataset differs significantly from all the pre-trained datasets.

My question is that if I want to use OV-DINO for fine-tuning, are there any techniques or tips to help me bridge the gaps in resolution, data amount, object size, and category names (or just bridge some of these gaps)? Thanks a lot!

[wanghao9610]

Hello @Fiorentinar , following are some advices for fine-tuning:

• More and larger input resolutions, OV-DINO default is [800, 1333], change them in custom_ovd.py.
• Fixe the text encoder, as your fine-tuning data scale is small, refer to issue 21.
• Keep the category name as simple as possible, as the pre-training category names are nouns or phrase of a single class, complicated text description may not matchd in the pre-training stage. For certain perfomance, you need to run experiments to test it.

Other settings no need to change.

[Fiorentinar]

Thank you very much for the tips you've provided. I'm particularly curious about the following two points:

1. In the scenario I described, would fixing the text encoder affect the fine-tuning of category labels(for those labels that are not commonly present in pre-trained dataset)?

2. For small object detection, do I need to modify the corresponding network structure, just similar to the changes from YOLOv8 to YOLOv8-p2 (where lower-dimensional backbone features are also fed into the neck and head)?

[wanghao9610]

@Fiorentinar Regarding your question:

1. You can run experiments with fixed and unfixed text encoders, and better performance is expected.
2. Modifying the framework will not load the pre-trained model, which may lead to worse results.

[Fiorentinar]

Thank you for your patient response; all my questions have been answered.