Waifu2x in real time

[Mickevincent]

Hello

I was told to check with you guys about my request too. I don't know if you are familiar with the Waifu2x upscaler and filter. It's both web-based and in program form. You can grab the program here

https://translate.googleusercontent.com/translate_c?depth=1&hl=en&prev=search&rurl=translate.google.se&sl=ja&u=https://github.com/lltcggie/waifu2x-caffe/releases&usg=ALkJrhivu32FwUwuPxodRAFRTS4bRFPimA

And here is a GLSL impementation.

https://github.com/ueshita/waifu2x-converter-glsl

My request is originaly to have this kind of upscaler and filter mainly for PS1 emulation since those games were rendered in really, really low res. Here is some examples I have done by upscaling original backgrounds from FFVII and FFIX.

FFVII http://i.imgur.com/X27lVJ5.jpg

FFVII http://i.imgur.com/fYwxIBY.png

FFIX http://i.imgur.com/zR1kjlE.jpg

FFVII http://i.imgur.com/CzRczDb.jpg

And here's another batch of FFVII pictures http://imgur.com/a/zzdBm

Well, as you can see this is probably the best option for making these games look as "hd" as possible. And it do looks really nice.

So, is there any possibility that we sometime can have a Waifu2x filter for realtime gaming? Please /Mike

[i30817]

This scaler uses neural networks. I seriously doubt it will ever get 'real time' in the next 20 years. At most, a texture replacement process (like ppsspp and dolphin) could be implemented and the work done off-line (it would be cute if it could done automatically too, by playing through the game once, extracting the textures, and then process them all and play with the new ones).

[simias]

Yeah I've looked a bit into that and I'll need to have some clever code to piece FF's prerendered together in a single texture. The game draws it as a series of tiles:

I still haven't figured out what's the best way to handle texture dumping and replacement, palettes make it more complicated than it ought to be. Anyway, I'll consider it eventually but it's a bit too early for that, I'm closing this until texture replacement and/or custom filters are implemented. Then we'll have all the time to experiment with it.

[i30817]

Texture caches and texture replacements often interact badly too. Dolphin has that problem, where unless you choose the 'safe' option for the texture cache, you'll likely have outdated replacements with its weak hashing to id those replacements originals (i think that's the problem, not know).

[simias]

Yeah, for the time being I managed to do without any cache but in order to implement texture replacement I'll need a cache of sorts to substitute the portions that are overloaded. It's going to be tricky.