Train YOLOv3 with FLIR images (IR camera)

[isra60]

Hi I want to improve my yolo detections with ir camera feed. Now we are using the standard yolo3.weights file and obviously is not as good as with a visible camera.

I know about this https://www.flir.com/oem/adas/adas-dataset-form/ from flir that contains Boxes from

1. Person (28,151)
2. Car (46,692)
3. Bicycle (4,457)
4. Dog (240)
5. Other Vehicle (2,228)

So this are categories which are already in the coco dataset which is yolo already trained

Now I have read about train with my own custom objects here https://github.com/AlexeyAB/darknet/blob/master/README.md#how-to-train-to-detect-your-custom-objects

But it seems with this retrain yolo will forgot to recognize a clock or a chair and then only will know about the FLIR dataset categories. So I don’t want that. I want that yolo knows that people, dogs etc also could seen as a thermal image but without losing its previous training

How could this can be done? Maybe just putting all the coco classes and only give to the train the thermal ones ? Or I will have to download the entire coco dataset and add the thermal images??

Thanks in advanced

[AlexeyAB]

@isra60 Hi,

Or I will have to download the entire coco dataset and add the thermal images??

Yes, you should train by using both COCO+FLIR datasets together.

• just class_id of Person, Car, Bicycle, Dog , Other Vehicle should be the same in labels from COCO and FLIR. So you can continue training by using pre-trained file yolov3.weights with flag -clear in training command

• Or you can add new classes, so

  • there will be normal-Person from COCO with class_id = 0 https://github.com/AlexeyAB/darknet/blob/master/data/coco.names#L1

  • and class_id = 80 for infrared-Person from FLIR, etc...., but in this case for training you should use yolov3.81.conv pre-trained file instead of yolov3.weights, that you can get by command: https://github.com/AlexeyAB/darknet/blob/9e9b2c493630a8ebd05d8732f0eb6b12a078bb88/build/darknet/x64/partial.cmd#L27

[isra60]

just class_id of Person, Car , Bicycle , Dog , Other Vehicle should be the same in labels from COCO and FLIR. So you can continue training by using pre-trained file yolov3.weights with flag -clear in training command

Maybe this is the best option. But If I am not wrong yolov3.weight file already is trained with COCO dataset right?

So why I have to download all the folders for each COCO category??

And can I only train with the 5 categories with the visible COCO images and IR images.. so I will have a YOLO for 5 categories right?

[AlexeyAB]

And can I only train with the 5 categories with the visible COCO images and IR images.. so I will have a YOLO for 5 categories right?

Yes, but you wrote that you don't what that.

But it seems with this retrain yolo will forgot to recognize a clock or a chair and then only will know about the FLIR dataset categories. So I don’t want that.

[isra60]

Yeah I know but maybe it takes too long... I only have a 1060...

[xevolesi]

Hi @AlexeyAB.

Yes, you should train by using both COCO+FLIR datasets together.

• just class_id of Person, Car, Bicycle, Dog , Other Vehicle should be the same in labels from COCO and FLIR. So you can continue training by using pre-trained file yolov3.weights with flag -clear in training command

what does '-clear' flag do and why i should use it in order to detect objects of same classes in both domains: visible and flir?

Can you also provide some advices for training YOLO-like models only on FLIR images? I've tried YOLOv3-Tiny, YOLOv3 and YOLOv3-spp-pan-scale. mAP results are:

• Tiny-YOLOv3: 40.44%;

• YOLOv3: 52.44%;

• YOLOv3-spp-pan-scale: 58%;

Can you advice something to improve this results?

Also what about color augmentation? Should i turned it off?