search

junyanz / pytorch-CycleGAN-and-pix2pix

Image-to-Image Translation in PyTorch
Other
23.09k stars 6.32k forks source link

IR to RGB method help #1436

Open songyn95 opened 2 years ago

songyn95 commented 2 years ago

I have paired IR and RGB traffic images (video sequences). My goal is to achieve IR colorization at night. The effect is the same as that of RGB in the daytime. I have tried many methods, such as cyclegan, pix2pix, pix2pixHD, vid2vid, few shot vid2vid, etc.

Since there are many traffic scenario, the traffic scenarios in different areas are different, and my dataset scenarios are limited, I would like to ask which method is the most appropriate and has the best generalization ability? Among the results of my current attempts, pix2pixHD works well, but after changing the scene, the effect is not good. Do you have any reasonable suggestions? Thank you very much for your reply！！

junyanz commented 2 years ago

Most of the methods tend to overfit the training set if you have limited data. You may consider applying data augmentation to combat model overfitting.

songyn95 commented 2 years ago

@junyanz So which method (including but not limited to) do you think is more suitable for the scenario I described？

junyanz commented 2 years ago

It may not be so related to the models you are using. You need to either experiment with different types of data augmentation or expand your dataset.

songyn95 commented 2 years ago

At present, I want to color infrared video. Is there any good way?

junyanz commented 2 years ago

You can try fewshot vid2vid. If you have a smaller dataset, you need to reduce the network's capacity as well. It's hard to make any above methods work out of the box.