search

MathiasGruber / PConv-Keras

Unofficial implementation of "Image Inpainting for Irregular Holes Using Partial Convolutions". Try at: www.fixmyphoto.ai
MIT License
909 stars 252 forks source link

How do I start training from scratch? #38

Open ghost opened 5 years ago

ghost commented 5 years ago

Sorry if this is a stupid question, I'm a newbie. I've tried reading the notebooks and looking at 'Issues', but I don't quite get how to get this thing to work. Here are some issues I'm facing:

  1. If I don't want to use pre-trained model and I want to start training from nothing, what should I change from the codes? Are the pre trained models requirements to run the program?
  2. What kind of pictures should I put into training, validation, and testing sets?
  3. If I want the program to predict an image (e.g. removing people from a photo), what do I do? Do I just use photo editing software to highlight people with a brush and run it on Step5? If so, what color?

Thank you

leafchenh commented 5 years ago

I have the same problem as you. Have you made any progress

ghost commented 5 years ago

I have the same problem as you. Have you made any progress

Unfortunately, no. I am working with a really specific dataset and I don't want to contaminate my data by using pre-trained models. I am new to Keras and Tensorflow. Although I can kinda understand how the code works, I don't know how to modify it without ruining it.

It would be great if someone made a simplified version of this, so more people can use it. I will try to make one once I figured out how to get it to work

NR222 commented 5 years ago

I am also no expert, but try to use:

1.in step 4 change into:

model = PConvUnet(vgg_weights='./data/logs/pytorch_vgg16.h5')

model.load(INITIAL_WEIGHTS_1)

the pytorch_vgg16.h5 can be downloaded from https://drive.google.com/file/d/1HOzmKQFljTdKWftEP-kWD7p2paEaeHM0/view .

2.I would try to insert premade masks( covering Humans ), which You can than use in training the model, but to modify the code for that is much more complicated, I am now trying something similar, but I just start the training so result is unkown.

3.Yes step 5 is correct, mayby You don't need to train from scratch, why don't You just try and use pretrained model ? and run step 5 only,with masks on those people( RGB=(0,0,0,0) )

leafchenh commented 5 years ago

Notebook 5 references "random_mask" which isn't available. I was able to get it to work by using 'MaskGenerator' But it tells me there's no shape

ghost commented 5 years ago

Notebook 5 references "random_mask" which isn't available. I was able to get it to work by using 'MaskGenerator' But it tells me there's no shape

put MaskGenerator()._generate() on that.

ghost commented 5 years ago

Sorry if this is a stupid question, I'm a newbie. I've tried reading the notebooks and looking at 'Issues', but I don't quite get how to get this thing to work. Here are some issues I'm facing:

  1. If I don't want to use pre-trained model and I want to start training from nothing, what should I change from the codes? Are the pre trained models requirements to run the program?
  2. What kind of pictures should I put into training, validation, and testing sets?
  3. If I want the program to predict an image (e.g. removing people from a photo), what do I do? Do I just use photo editing software to highlight people with a brush and run it on Step5? If so, what color?

Thank you

  1. Of course, only if you could get your own pre-trained model for the perceptual loss and style loss. Then, you can load model weights from that. Otherwise, your model give you unexpected results with blurriness or checkerboard artifacts.
  2. Generally, you can use Imagenet dataset.
  3. Which one would you use on ? In common, you can apply it on HTML(JS) with python Flask like NVIDIA Online Demo. or, call the python executive application library.

Best regards,

fivoskal commented 4 years ago

hi, i am also having trouble using notebook - 'Step 4 - Imagenet Training'. Specifically the part below. Could someone provide me with clarification to this? # Instantiate the model model = PConvUnet(vgg_weights='./data/logs/pytorch_vgg16.h5') model.load(r"C:\Users\Mathias Felix Gruber\Documents\GitHub\PConv-Keras\data\logs\single_image_test\weights.10-0.89.h5")

I am guessing './data/logs/pytorch_vgg16.h5' should be change to provided file path './data/logs/pytorch_to_keras_vgg16.h5'

But what argument should I provide in model.load()?

yankee1996 commented 4 years ago

where is the "single_image_test\weights.10-0.89.h5"?

sariva03 commented 3 years ago

hi, i am also having trouble using notebook - 'Step 4 - Imagenet Training'. Specifically the part below. Could someone provide me with clarification to this? # Instantiate the model model = PConvUnet(vgg_weights='./data/logs/pytorch_vgg16.h5') model.load(r"C:\Users\Mathias Felix Gruber\Documents\GitHub\PConv-Keras\data\logs\single_image_test\weights.10-0.89.h5")

I am guessing './data/logs/pytorch_vgg16.h5' should be change to provided file path './data/logs/pytorch_to_keras_vgg16.h5'

But what argument should I provide in model.load()?

hi, i am also having trouble using notebook - 'Step 4 - Imagenet Training'. Specifically the part below. Could someone provide me with clarification to this? # Instantiate the model model = PConvUnet(vgg_weights='./data/logs/pytorch_vgg16.h5') model.load(r"C:\Users\Mathias Felix Gruber\Documents\GitHub\PConv-Keras\data\logs\single_image_test\weights.10-0.89.h5")

I am guessing './data/logs/pytorch_vgg16.h5' should be change to provided file path './data/logs/pytorch_to_keras_vgg16.h5'

But what argument should I provide in model.load()?

any solution?