FSR 2.0

[Repsycl3d]

It's out! Can we get this plz?

[fholger]

Afraid not. Same as DLSS, this one requires deep engine integration and can't just be added generically to any game.

[Espionage724]

FSR 2.0 is now open-source: https://gpuopen.com/fidelityfx-superresolution-2/

[andrewpros]

@Espionage724 still needs engine integration, at a minimum color & depth buffer & motion vectors, maybe some games would be possible, but still lot of work and very unrealistic, its also changes the project initial goals, its easy with fsr or nis as it only needs the image, nothing else

[GreenMan36]

Reading the docs it seems a "Reactive mask" is used for area's with transparency.

As some areas of a rendered image do not leave a footprint in the depth buffer or include motion vectors, FSR2 provides support for a reactive mask texture which can be used to indicate to FSR2 where such areas are. Good examples of these are particles, or alpha-blended objects which do not write depth or motion vectors. If this resource is not set, then FSR2's shading change detection logic will handle these cases as best it can, but for optimal results, this resource should be set. For more information on the reactive mask please refer to the Reactive mask section.

I'm probably wrong but if you provide dummy data for the other buffers and use this to mark everything as transparant, it might provide interesting results?.. or terrible/broken ones.

That would mean dummy data for the Depth buffer and Motion vectors and only using the color buffer with automatic exposure.

Feel free to correct and educate me.

They do say

It should be noted that it is unlikely that a reactive value of close to 1 will ever produce good results.

[fholger]

That mask is for preventing the algorithm from doing the wrong thing in parts of the image. But there isn't much the algorithm can actually do with those areas (otherwise we wouldn't even need the complicated setup of providing depth and motion vectors), so masking the entire image will effectively prevent FSR from doing anything meaningful at all.

[andrewpros]

@GreenMan36 Actually it could be doable to provide depth & color buffers, this maybe would provide interesting results, mask is the most irrelevant one.

The real challenge would be motion vectors as not every game has it or exposes it, most vr games or maybe almost all dont have it i guess.

If something can be dummy empty data i would say motion vectors and check the results without them.

But this got me thinking, maybe there is a way to approximately create some motion vectors from depth or stencil buffer, probably not easy and still not that great, but maybe better than empty data.

But still, all of this is additional work.

[fholger]

Motion vectors are required precisely because you can't properly tell how fast individual pixel data moved between frames. Although there are TAA implementations that work without motion vectors, it severely limits what they can do. Given that VR games inherently add a lot of head motion to the rendered images, good temporal anti-aliasing is especially important, and so any attempt to feed the algorithm with subpar data is not going to do anyone any good.

[andrewpros]

I dont know what will happen with blank motion data pushed to FSR 2.0, didn't test it, dont know the result so i will not speculate, either it will work but worse result or won't work completely.

Also im not sure how much head motion has to do with it, FSR/DLSS wants the motion vectors of things in the scene moving, not sure this also means camera or compensating if things are moving in the scene and u are moving the camera too, probably both.

But we can get the head movement from the game, the scene objects movement is the issue as it wont be there, best case scenario is somehow emulate it or not, as a said maybe it works without with worse results, but worse than FSR 1.0?, who knows.

Im just curious and theorize, maybe there is a game and a way to test it once.

[fholger]

My point wasn't about the head movement's contribution to the image movement, because that is indeed available to the algorithm via the view transform. It was that the general increased motion in the image due to constant small head movements makes good temporal anti-aliasing all the more important compared to the more stable camera views in flat games. Therefore you need the algorithm at its best, which you won't get if you don't supply motion vectors.

[andrewpros]

And as is said, im unable to test it, so i cant speculate of how much the difference would be, cuz it could be still better than fsr 1.0.

[wazzzaatosh]

hi, this guy did it for CP2077: https://github.com/PotatoOfDoom/CyberFSR2 via DLSS though

[GreenMan36]

Simply put; FSR 2.0 REQUIRES more information FROM THE GAME which the SteamVR Compositor simply does not have. It is not possible to implement FSR 2.0 into the VR Performance Toolkit unless you prefer lower quality for less framerate than just running at a lower resolution.

@wazzzaatosh no... the CP2077 mod doesn't implement FSR 2.0 ??via?? DLSS, it replaces DLSS. The reason a mod like that works is because its implemented on a per-game basis, deep into the game's rendering pipeline where you do have the information required to implement FSR 2.0.

The SteamVR Compositor doesn't have this information and the result of faking data, like the FSR 2.0 documentation says is not going to be any good or usable. Unless you prefer warping, aliasing, blur and a generally unstable image (think similar artifacts to constant reprojection, although not the same, its a good comparison). Which is certainly worse than FSR 1.0 especially in VR.

@fholger maybe its a good idea to close the issue with a clear statement of why FSR 2.0 won't/can't work? And to correct anything that's said wrong, if said wrong.

[andrewpros]

I cant believe it but someone has actually try it, FSR 2.0 with motion vectors disabled and it is absolute disaster https://www.youtube.com/watch?v=PAp3WDzv7W8

So yeah, only way is to get the motion vectors from the game or create them somehow from other game data if the game is not already providing it, in any case this is far beyond the scope of this project.