search

pfnet / PaintsChainer

line drawing colorization using chainer
https://paintschainer.preferred.tech/
MIT License
3.77k stars 561 forks source link

Training own models #143

Closed Schenng closed 5 years ago

Schenng commented 5 years ago

Hi @taizan.

Thank you for the great project.

I have successfully run PaintsChainer with your supplied models, but cannot train my own models. I am running into the same "green" issue described in #84. (I am starting a new issue because #84 is over a year old).

I have tried many things to solve it such as:

The following are the output images after training a very small dataset of 30 images from Pixiv. The more epochs, the more green the picture becomes.

20 Epochs 40 Epochs 100 Epochs

I also had to change the code in train_128.py on like 171.

t_out[i, 0, :] = xp.asarray(batch[i][1]) -> t_out[i, :] = xp.asarray(batch[i][1])

because of the following error: 

  File "train_128.py", line 171, in update_core
    t_out[i, 0,:] = xp.asarray(batch[i][1])
  File "cupy/core/core.pyx", line 1553, in cupy.core.core.ndarray.__setitem__
  File "cupy/core/core.pyx", line 3207, in cupy.core.core._scatter_op
  File "cupy/core/elementwise.pxi", line 820, in cupy.core.core.ufunc.__call__
  File "cupy/core/elementwise.pxi", line 404, in cupy.core.core._get_out_args
ValueError: Out shape is mismatched

Setup 1 (Personal machine): Ubuntu: 16.04 Python: 3.5 GPU: NVIDIA Titan V CUDA: 9.1 cuDNN: Tried 7.3.1 & 7.3.1 Chainer: 4.4.0

Setup 2 (Google Deep Learning VM): Debian: 9.5 Python: 3.5.3 GPU: NVIDIA Tesla K80 CUDA: 9.2 cuDNN: 7.2.1 Chainer: 4.5.0

Image array after reading RGB2YUV: [[[158 82 147] [158 82 147] [158 82 147] ..., [254 128 129] [255 128 128] [255 128 128]]

[[159 82 147] [159 82 147] [159 82 147] ..., [248 124 130] [250 124 130] [252 124 130]]

[[159 82 147] [159 82 147] [159 82 147] ..., [240 122 130] [243 122 130] [246 122 130]]

..., [[176 97 135] [199 98 135] [211 100 136] ..., [157 82 147] [156 82 147] [149 82 147]]

[[164 85 145] [164 85 145] [166 86 144] ..., [163 82 147] [167 82 147] [165 82 147]]

[[160 79 152] [163 79 152] [160 78 151] ..., [185 82 147] [196 82 147] [199 82 147]]]

Any advice would be highly appreciated. Thoughts @lllyasviel?

Thanks. Simon

taizan commented 5 years ago

Hi, can you reconstruct original image using only 1 data without adversarial loss? If it could not possible , there is some problem on environment.

lllyasviel commented 5 years ago

If you are researcher, and you want to reproduce paintschainer for comparison, I highly recommend to directly use paintschainer web service as your baseline.

If you are currently learning deep learning and want to colorize anime sketches for fun, I think pix2pix is a better start:

https://github.com/pfnet-research/chainer-pix2pix https://github.com/affinelayer/pix2pix-tensorflow

In fact you can good results very similar to paintschainer V1 from pix2pix, if you have a large enough datasets, after weeks of training. Pix2pix codes are much better prepared and much more easy to read.

Schenng commented 5 years ago

Hi @taizan Sorry for the delay. I did not have success reconstructing the image using only 1 data without the adversarial loss. I guess it's an environment problem...

@lllyasviel I have started exploring pix2pix and it seems like a good alternative. Thank you for the advice!