search

pfnet / PaintsChainer

line drawing colorization using chainer
https://paintschainer.preferred.tech/
MIT License
3.77k stars 561 forks source link

A new way to get sketch from paints #115

Open lllyasviel opened 7 years ago

lllyasviel commented 7 years ago

I am working on my paper and I implemented a new nn to get sketch from paints. Unlike the "lnet" in paintsChainer, It can drop useless details and add lines to make the sketch looks better. I think it may be of help. https://github.com/lllyasviel/sketchKeras

abbychau commented 7 years ago

Great work! but I think you should at least show how the model to be trained.

taizan commented 7 years ago

Looks nice. My published L-net model is trained for 128x128 YUV input to use as apart of loss function. In your page the input of L-net is maybe RBG. In my environment, the output of 128_lnet and 512_lnet is below. 128_lnet ・looks noisy & one line is split, because miss much of scaling factor 512_lnet ・looks more better output

lllyasviel commented 7 years ago

yes i use yuv channel for lnet. I am writing the training way now.

taizan commented 7 years ago

Oh, sorry for miss understanding. Anyway, I'm waiting for your detailed article about it.

lllyasviel commented 7 years ago

I think the main difference between our training is the the attitude torwards details. Take "eye" as an example, your lnet has the ability to convert the eye into sketch perfectly. However, when human draw a sketch for ACG, they always ignore the detail and only draw a circle. Hence the paintsChainer receives lots of sketchs with "cycle eyes" but the training data is "detailed eyes". My net tends to erase the content of eyes to make it looks like cycles.(However not reach the state of art but only a trend )

Here is a example

real eyes in a sketch drawed by human (cycle eyes):

qq 20170506153833

eyes in sketchs from your lnet (detailed eyes)

qq 20170506154002

qq 20170506154051

my sketchKeras tends to erase the contents of eyes (looks like cycle eyes)

qq 20170506154256

My net still can not erase perfectly now , but after threslod the sketch , the result is not the state of art but respective:

qq 20170506154633

qq 20170506154744

This pair shows the difference apprently:

yours:

qq 20170506154946

mine:

qq 20170506154933

This is only my opinion but I think it is the main reason why paintsChainer has no ability to handle "circle eyes" . such as:

qq 20170506155431

However I am not sure that my theory is right and can be of help. I am doing many experiments now, including w-cgan and ls-cgan.

taizan commented 7 years ago

yeah, reconstruction of highlight of the eyes are one of the improvable point of PaintsChainer. I think many way of extraction line is important for data augmentation.

oasfuyou commented 4 years ago

where i can find the L-NET example