search

Artoriuz / ArtCNN

Super-Resolution Convolutional Neural Networks as GLSL shaders for mpv
MIT License
109 stars 2 forks source link

Appropriate content for this scaler #1

Closed Zabooby closed 8 months ago

Zabooby commented 8 months ago

I'm guessing by the name that this scaler is recommend only for anime, am I wrong or can it be used for live action too?

Artoriuz commented 8 months ago

It's trained on Manga109 so it's "optimised for anime" to some extent...

It should be fine on anything though.

Jules-A commented 8 months ago

I'm only tested it on anime so far and it works there with the exception of aliasing (which it performs very poorly on) and excessive thinning which is mainly only noticeable at smaller base resolutions. I can't see why it wouldn't work well on non-anime content if it's of high quality already.

Pretty sure I had the exact same issue with the LineArt version of FSRCNNX though to a lesser extent.

Zabooby commented 8 months ago

Thanks, I'll check it out.

deus0ww commented 6 months ago

Do you plan to train with live/non-anime content? Maybe a LiveCNN?

Artoriuz commented 6 months ago

Do you plan to train with live/non-anime content? Maybe a LiveCNN?

To be fair not really. There are way too many variants by now and redoing all of this for live action would be a pain in the ass. At most I can re-add the DIV2K C4F16 model but that's it.

Artoriuz commented 6 months ago

Have fun: https://github.com/Artoriuz/ArtCNN/blob/main/ArtCNN_C4F16_DIV2K.glsl

deus0ww commented 6 months ago

No pressure and thank you.

It would be nice to have something like/better than fsrcnnx that's still maintained/improved but I'll take whatever I can get. If you ever decide to take this up, I don't think you need to gen all variants for live content. Just the main ones, at 8,16,32, would be great.

ghost commented 6 months ago

@deus0ww Did you compare ArtCNN_c4F16_DIV2k and fsrcnnx? which one is best for live stuff

deus0ww commented 6 months ago

@deus0ww Did you compare ArtCNN_c4F16_DIV2k and fsrcnnx? which one is best for live stuff

Subjective testing in progress...

ghost commented 6 months ago

Okay, can you share the result of comparison after they are done?

Artoriuz commented 6 months ago

Just keep in mind this is the normal variant which doesn't sharpen or denoise the content.

deus0ww commented 6 months ago

Just keep in mind this is the normal variant which doesn't sharpen or denoise the content.

That's preferable but it does make comparison with fsrcnnx more difficult.

deus0ww commented 6 months ago

My subjective 'review' - FSRCNNX 16 vs ArtCNN 16 DN/DS:

What I'm using:

Zabooby commented 6 months ago

My subjective 'review' - FSRCNNX 16 vs ArtCNN 16 DN/DS:

* Sharpness: FSRCNNX is between the DN and DS variants in sharpness. ArtCNN DS is sharpest.

* Denoise: ArtCNN DN/DS denoise more heavily than standard FSRCNNX, but much less than FSRCNNX LineArt variant.

* Aliasing/Blocking: ArtCNN is noticeably better.

* Ringing: FSRCNNX is much worse (maybe by amplifying ringing in the source).

* Line Thinning: ArtCNN has thicker lines than FSRCNNX.

What I'm using:

* For live videos, I'm now using ArtCNN DS when it's the only scaler, and DN when it's followed by another scaler (depends on scale factor).  Combining the DS variant with another scaler is too oil-painting-like.

* For animated videos, I'm using ArtCNN DS for all scale factor.

Did you do your tests on high or low bitrate videos?

Also does nnedi compare well to ArtCNN or FSRCNNX, they aren't included in Artoiuz's comparisons?

deus0ww commented 6 months ago

Did you do your tests on high or low bitrate videos?

All sorts of bitrate. This is my general thoughts. For very clean sources (like images), I do switch to the SH and standard/div2k variants.

Also does nnedi compare well to ArtCNN or FSRCNNX, they aren't included in Artoiuz's comparisons?

I haven't used NNEDI in years so I don't know.

Artoriuz commented 6 months ago

Also does nnedi compare well to ArtCNN or FSRCNNX, they aren't included in Artoiuz's comparisons?

NNEDI3 was removed from the comparison because it introduces a half-pixel shift which makes all metrics go to shit. Fixing the shift blurs the image which defeats the purpose of using a doubler.