search

junyanz / pytorch-CycleGAN-and-pix2pix

Image-to-Image Translation in PyTorch
Other
22.68k stars 6.27k forks source link

pix2pix for Rgb to thermal image translation #1037

Open durgesh17media opened 4 years ago

durgesh17media commented 4 years ago

has anyone performed pix2pix approach for RGB to Thermal image translation? I am getting very bad results while testing. anyone can help? I used this command for testing: python3 test.py --dataroot ./datasets/combine_faces/ --name face4_pix2pix --model pix2pix --direction AtoB

results: Real A

1100_real_A

Real B 1100_real_B

FakeB 1100_fake_B

junyanz commented 4 years ago

If you have many training images, you can try pix2pixHD or SPADE.

durgesh17media commented 4 years ago

thanks for the reply @junyanz sir , I have 1400 rgb and thermal image pair 128*128 pixel. will it be ok for pix2pix hd or spade? any tips for parameter tuning while training?

junyanz commented 4 years ago

You can use the default parameters. I also recommend that you use random cropping (e.g., --load_size 143 --crop_size 128 and flipping.

durgesh17media commented 4 years ago

This you are telling for pix2pixHD?

junyanz commented 4 years ago

You can try both. Pix2pixHD might have different naming conversions for these flags.

durgesh17media commented 4 years ago

Still getting bad results after --load_size 143 --crop_size 128 . Training command :python3 train.py --dataroot ./datasets/combine_faces/ --name face_new12_pix2pix --model pix2pix --direction AtoB --netG unet_128 --n_epochs 200 --n_epochs_decay 200 --preprocess none --load_size 143 --crop_size 128

and one more thing ,the model only converts rgb to thermal for only the test folder images in combine_faces, but when I am giving random images of mine or someone else's after resizing it to the trained data dimension , the model fails miserably. Fake B 1004_fake_B real A 1004_real_A real B 1004_real_B 1005_fake_B 1005_real_A 1005_real_B 1001_fake_B 1001_real_A 1001_real_B 1002_fake_B 1002_real_A 1002_real_B 1003_fake_B 1003_real_A 1003_real_B @junyanz please guide.

junyanz commented 4 years ago
  1. Not sure how to further improve the quality. Maybe you can try different generators such as --netG resnet_6blocks or --netG resnet_9blocks. But there is no guarantee.
  2. If your training data only contains faces of one person or a few persons, it may not work for your face. 2. Also for your new test data, you need to align and crop it before feeding to the network. 3. For a single test image, you need to use --model test. See this script for a reference.
durgesh17media commented 4 years ago

hello @junyanz , can you tell me what is this .ipynb_checkpoints folder in PyTorch-CycleGAN-and-pix2pix/results/face_new6_pix2pix/test_latest/images???? why it is being created and what is the use of this?

durgesh17media commented 4 years ago

@junyanz please respond on the above question.

junyanz commented 4 years ago

This post explains the ipytnb_checkpoints. It might be caused by the creation of a iPython notebook.

sobhani commented 4 years ago

Do you think if we play with the learning rate to higher (?) or lower(?), might affects the performance of the networks having better-generated results? else what could help for generating better outcomes? either in oix2pix or Pix2pixHD?

junyanz commented 4 years ago

Not sure. It all depends on your application and datasets. Quite hard to predict which learning rate is the best in advance.