Thank you for this masterpiece! Some Feedbacks and discussions.

[2ji3150]

Appreciate

First of all, Thanks this great piece of Masterpiece. Great Job xinntao, Great job Tencent ! Absolutely Fantastic. We all see how strong and how you contribute the community.

As a anime-game-fan (which is call 2jigen culture) , I was first impressed by waifu2x ( I believe it was release on 2015 arround) After 7 years, The cunet ver of waifu2x is still the best on the anime scaling. I was belived that waifu2x is almost the ai upscaling limit. But now, You bring me to the future. A really another dimension! Welcome 2021. Another milestone!

Feedback on anime model

Let's talk about the feedback on the real-esgran. some of them are note in the feedback.md I believe. But I'll going to write all again.

Issues

• Foreground and background recongnization and separation Current model can not detec the context in the image and just mark anything as sharp as posible. That make the 2D illustration lose there depth and shaddow.

• Not good at animations processing Current model does not use the pre-frame and next-frame to analize and optimize the processing. And due to the depth information loss and the overpower noise clearing, it would remove lots of thing like memory noise or motion blur. which should not be remove. Maybe animation processing need to make some more much bigger model to combind the ability to ① analize scenario and artifact ② again - background recongnization and separation ③ motion analysis This should be very very challenging I think. For realtime processing , I doubt it will be come true. Maybe tensor core can help us?

• And for the paint style issue, I think maybe it is better to seperate anime model and the illustration (CG) model. because most of the time anime and illustration are much different. Use both of the data to training may be challenging to process both to a good resualt. In my opnion anime model should Focus on the anime. And Extend to animation processing.

• If we make an illustration (CG) model , Maybe it could be bigger than anime model, cause there are many paint styles. We should keep the original style in default and give option to some other styles and noise remove levels.

Ideas

• Scatch model can benifits lots of illustrators. And I think it can be much smaller maybe.
• Manga (comic) model. I think Upscale manga is much much challenging than anime and illustration . The current model just remove all of the Screentone and other effects. Or Maybe I have to wait another 10 years. I don't know.

Questions

• What is under desinged and are you going to keep update this project? Do you have a roadmap or plain.
• How to help this project. Is this kind of project able to do something like Distributed calculation ( like project leela-zero or katago) to contribute caculation resource?
• Are there any plan for performance optimization like using tensor-core?
• Is this nearly the limit of real-esrgan? Are there more rooms to improve ? I feel there are just a few steps behind near truely ground truth on anime/illustration .

---

AI upscaling is really really amazing. It is nearly perfect. But I guess it may take another 10 years to completely deceive human eyes.

[xinntao]

Wow 👍 Very thanks for your feedback and suggestions. Let me have a close look 😄