search

TencentARC / AnimeSR

Codes for "AnimeSR: Learning Real-World Super-Resolution Models for Animation Videos"
Other
327 stars 34 forks source link

[Question] Roadmap and future Support #8

Open Jumaron opened 1 year ago

Jumaron commented 1 year ago

Hi Guys, it's me again, i really started liking your project and was questioning myself if you guys want to keep working/updating it for the future since i have not found any roadmaps or information about that.

Really love it keep it going!

Jumaron commented 1 year ago

Also is it possible to add TTA?

Zarxrax commented 1 year ago

I would add to this topic, if the AnimeSR team is serious about creating a model for restoring Animation, the current degradation pipeline is insufficient, because it does not accurately model the real issues in low quality video.

Most of the animated content that people want to upscale and restore is older content from DVD, VHS, NTSC broadcast, etc. The video contains issues like Dot Crawl, Rainbows, Color Bleed/Chroma noise, Edge Enhancement/Halo, MPEG-2 compression, among others. I have trained effective Real-ESRGAN models on several of these issues so I know they are possible to fix. I kindly recommend that you focus on these types of analog video degradations in order to make the model more effective for real world use.

Jumaron commented 1 year ago

I would add to this topic, if the AnimeSR team is serious about creating a model for restoring Animation, the current degradation pipeline is insufficient, because it does not accurately model the real issues in low quality video.

Most of the animated content that people want to upscale and restore is older content from DVD, VHS, NTSC broadcast, etc. The video contains issues like Dot Crawl, Rainbows, Color Bleed/Chroma noise, Edge Enhancement/Halo, MPEG-2 compression, among others. I have trained effective Real-ESRGAN models on several of these issues so I know they are possible to fix. I kindly recommend that you focus on these types of analog video degradations in order to make the model more effective for real world use.

Your right to this about all the things that need to be done in Order to fix it my current problems are the following:

Without TTA i get weird artefacts between each Frame that look not only weird but are so annoying especially on high End Models like Anime_B6 from RealESRGAN that it kills me without.

Now comes the big Problem, as example Anime_B6 i very very slow especially with TTA, but the quality after it is just Mindblowing good.

Thats why i always search for new tech that can either have a Model with the Same Quality as Anime_B6 but lower Size or uses things like TensorRT to accelerate the use. Because rendering it in the Normel way with TTA Anime_B6 takes for a 24min Anime around 10Hours for me (RTX 3070). Cutting it by a factor of 2-4x and it will be doable.

Thats why i'am so persistent for new Updates because i know, that we can achiev that with our current tech aviable. I'am also trying to use TensorRT right now for myself but i have Problem after Problem, but still going strong.

Thanks for reading that ultra long text and for your tips. (I'am German so sorry for the grammar)

Jumaron commented 1 year ago

I forgot to add that i saw a Project named enhancr, they use TensorRT, but building it yourself is the biggest headache i ever had because you need to set all your Paths yourself and the code is not that clean.

Would really love to talk about it with you if you want and have some spare Time. You can writte me here: Jumaron#0509

manik563A commented 11 months ago

I forgot to add that i saw a Project named enhancr, they use TensorRT, but building it yourself is the biggest headache i ever had because you need to set all your Paths yourself and the code is not that clean.