I was wondering why you fixed the input size of your network as well as the output size... untill I find that sketchKereas only accepts input in the size of 512*512. Is there any method to make sketchKereas to accept flexible input? Anyway, thanks a lot.

[rapidrabbit76]

thank you for interest in my project.

Q. Why fixed the I / O data shape ?

A. because of coloring stability. after training I tried flexible I/O Data Shape(is set None in Input Shape) but the result was so unstable(result was Suck...). perhaps it was because the thickness of the sketch was subtly different depending on the image size. so I tried to erode & dilate using "OpenCV" for the sketch with data augmentation in training pipeline, but training result was also unstable. ㅜㅅㅜ if you have an idea of using flexible I/O data shape, please tell me. :)

Q. Method of flexible I/O data shape in SketchKeras

A. I'm sorry, but I'm not sure about this part. I am working on version 2.0. This time, I will use xDoG. there is LineDistiller, another line extractor project using machine learning

[papahxj]

Thanks a lot for your reply. This is my frist time to forth an issue on github and I don't fully understand how does it work. That's why I left the issue description empty. I'm sorry for my rudeness.

I thought at first that you create your dataset with all 512 * 512 pics using sketchKereas, so that you fixed your I/O shapes. But I was wrong, it's not the case.

I don't think fixing the I/O shapes and using OpenCV to scale pics is a good idea. One reason is that once we scale our pics, the characteristics of pics will change. Maybe there are some common characteristics in different line pics, but because of scaling, they become different characteristics. So the network fails to learn. Even worse, scaling may destroy lines. If we scale a 512 1024 pics, with lines in 2 pix wide in it, into 512 512, the lines may become less visible.

For unstable color results, I guess there are two reasons. One is that the using of pix-to-pix loss you add to the generative model. As far as I think, in the two-stage-colorize system, the first stage (draft model) is to generate colors, and it‘s ok if some mistakes occur. Then the second stage is working to correct the mistakes. But if you add pix-to-pix loss to the first stage, you added correction task to the gen model. This may cause the gen model learn specific patterns of pics, and make it works perfectly on training data but fails on testing data. Another reason I think is the improper dataset for training. I'm not sure for this point, but I think it's hard for our network to learn colorizing if our dataset is Danbooru or anything like that. The pics in those datasets are so noisy and are in different painting styles, which is not good for our network.

In fact, I think using correct datasets is the key for auto-sketch-colorize tasks. In my own work, I'm trying to use cartoon video snapshots as my dataset. If it works, I will tell you.

And, thanks a lot for the information of line extractor. Thanks ^ ^

[rapidrabbit76]

thanks for your reply :)

I agree with the importance of dataset for training I'm also trying to use anime(4:3 rate, not cartoon) video capture

I don't understand the loss part. Can you explain it in detail ?(this is my first ML project.) Is it a problem using pix-to-pix loss in the first stage(draft model)? When the results were bad, there was a tendency for colors to not be painted in the first stage(draft model).

I'm thinking of using "content loss" instead of l1_loss in pix-to-pix loss. Do you have any other loss of image generation?

I'm sorry for my poor english skills. Thanks again for the reply. :)  

[papahxj]

For the loss part, that was only my assumption. I haven’t done any experiments on this yet. For more information about loss of GANs, I recommend this. Maybe you can try different ways and it’s sure one of them will work.

Good luck for both of us. ^ ^ (BTW your English is quite good!)