search

libretro / RetroArch

Cross-platform, sophisticated frontend for the libretro API. Licensed GPLv3.
http://www.libretro.com
GNU General Public License v3.0
9.81k stars 1.78k forks source link

AI Upscaling integration (ESGRAN for example) #11328

Open romulasry opened 3 years ago

romulasry commented 3 years ago

First and foremost consider this:

Description

AI upscaling as a video setting [NO AI upscaling]

Expected behavior

Nice AI upscaled for al games. [What you expected to happen]

Actual behavior

Not implemented yet [What is actually happening]

Steps to reproduce the bug

  1. [First step]Run RetroArch
  2. [Second step]Go to Settings > Video
  3. [and so on...] No AI upscale setting

Bisect Results

[Try to bisect and tell us when this started happening]

Version/Commit

You can find this information under Information/System Information

Environment information

hizzlekizzle commented 3 years ago

AI upscaling is typically slower-than-realtime. However, the NNEDI3 shader is an implementation of neural net-derived weights.

romulasry commented 3 years ago

How about some type of running it and caching it "before" it runs?

i30817 commented 3 years ago

That only works in cores that can dump games textures, and each console and core dumps them with their particular ideas and limitations. There is no standard for this that would work to recognize a 'texture' in all platforms, and there isn't even a need for one, since particular projects that are not automatic will always be better.

Just go to the pages where you can find texture replacement projects for particular emulators/cores (like the recent resident evil 2 and 3 project), there is no possible general solution here that isn't realtime, and realtime is impossible right now (and probably forever the way the world is going) and would be inferior anyway for various reasons.

romulasry commented 3 years ago

If it was standardized then there wouldn't be a problem. Good point though.

i30817 commented 3 years ago

That's the point, it can't be standardized because the input is too different from console to console (sometimes from game to game on certain older consoles). The reason why other runtime filters are standardized is because they're runtime and only need to operate on the final image that goes to the screen - this is too slow to be runtime, so it needs to dump textures.

romulasry commented 2 years ago

Would be nice if it included them by default: https://github.com/ZironZ/NNEDI3-Slang-Shaders

hizzlekizzle commented 2 years ago

it does: https://github.com/libretro/slang-shaders/tree/master/nnedi3

romulasry commented 1 year ago

This could be done in real time with Neuromorphic computing