[Overview] Models differences

YukiSakuma commented 8 years ago

I want to know the differences between the Ukbench, photo, RGB, and Y-model and what do you think is the best model in your opinion thanks..

nagadomi commented 8 years ago

Here is the results of internal benchmark. TL;DR: For photo , models/photo is best. For artwork, models/anime_style_art_rgb is best. EDIT: For anime(video), models/photo is best, I think.

dataset

photo_test: 300 various photos.
art_test  : 90 artworks (PNG only).

pre-trained model

anime_style_art_rgb: This model is trained on artworks (images are like pixiv or yande.re). anime_style_art (Y): This model is trained on artworks. The model converts luminance channel only. It is more conservative than RGB model. photo: This model is trained on kou's photo collections. ukbench: This model is trained on ukbench.

2x upscaling model

Dataset/Model	anime_style_art(Y)	anime_style_art_rgb	photo	ukbench
photo_test	29.83	29.81	29.89	29.86
art_test	36.02	36.24	34.92	34.85

The evaluation metric is PSNR(Y only), higher is better.

Denoising level 1 model

Dataset/Model	anime_style_art	anime_style_art_rgb	photo
photo_test Quality 80	36.07	36.20	36.01
photo_test Quality 50,45	31.72	32.01	32.31
art_test Quality 80	40.39	42.48	40.35
art_test Quality 50,45	35.45	36.70	36.27

The evaluation metric is PSNR(RGB), higher is better.

Denoising level 2 model

Dataset/Model	anime_style_art	anime_style_art_rgb	photo
photo_test Quality 80	34.03	34.42	36.06
photo_test Quality 50,45	31.95	32.31	32.42
art_test Quality 80	39.20	41.12	40.48
art_test Quality 50,45	36.14	37.78	36.55

The evaluation metric is PSNR(RGB), higher is better.

nagadomi commented 8 years ago

I am going to release new models/anime_style_art(Y only) model. It might be able to beat the RGB model in 2x.

YukiSakuma commented 8 years ago

Thanks what does it mean when using photo model? Can I use photo model for upscaling anime (video) screenshots or video frames? like this one? http://animescreencaps.com/

nagadomi commented 8 years ago

Yes. anime_style_art_rgb is specialized for digital illustration. Anime is made by video recording. anime_style_art_rgb sometimes makes weird contour line for Anime(includes video frames and screenshots).

source: anime anime_style_art_rgb: photo:

YukiSakuma commented 8 years ago

But I think in terms of noise reduction RGB is better than Photo for denoising video frames (anime)

Kohit commented 7 years ago

Hi, I'd like to know the differences between vgg_7 and upconv_7 in pre-trained models. thanks.

nagadomi commented 7 years ago

vgg_7 and upconv_7 have different network architectures. vgg_7: 1. 2x the input image with nearest neighbor upscaler. 2. Repair the image with CNN upconv_7: 1. end-to-end 2x with CNN (it uses deconvolution layer).

upconv_7 is 2~3x faster than vgg_7, and its average upscaling quality is better than vgg_7. Simply, upconv_7 is the new version, vgg_7 is the older version. But upconv_7 has checkboard artifact problem. It happens very seldom, though.

Kohit commented 7 years ago

Thanks. I've read the ESPCN paper. Is upconv_7's last layer like that (sub-pixel convolution)?

nagadomi commented 7 years ago

The purpose is the same, but the algorithm is different. I tried subpixel convolution before the ESPCN paper was published(github log) but eventually I chose deconvolution.

splinter21 commented 2 years ago

The purpose is the same, but the algorithm is different. I tried subpixel convolution before the ESPCN paper was published(github log) but eventually I chose deconvolution.

Why did you choose deconvolution eventually? Is there any cases for comparision?

nagadomi / waifu2x