RSX: Reimplement the vertex processing stage; Minor fixes

[kd-11]

• This set reimplements the RSX driver side of the vertex processing stage. Vertex attribute data is no longer processed on the CPU freeing up resources otherwise used up dealing with this task. This moves a lot of work over GPU side and slow iGPUs might become a bottleneck. Due to the increased complexity of the shaders, a preloaded shader cache system is also included to allow the shader compilation and linking to happen outside of gameplay. This fixes the hitching and stuttering ingame at the cost of longer loading times. The cache can be deleted from the context menu.
• Also included is a partial rewrite of the vulkan renderer to take advantage of parallel frame processing. The frame queue is now double-buffered improving performance and reducing stutter.
• Also includes several minor fixes that should fix Ni No Kuni on both vulkan and opengl
• Properly implement VSync control when using openGL (at least under windows)

I am aware that intel chips are missing output in some games and will continue investigating the matter. In the meantime, any other issues encountered with AMD or Nvidia GPUs should be tracked here.

[kamer1337]

Reduced performance in Tony Hawk's Project 8, fps drops to 0.5 a lot, stutters really bad for me

[kd-11]

You have to let the shader cache build first. WIthout it the graphics drivers sometimes struggle linking the large shaders. Its not a bug. The first run will be slow - no way around that.

[Illynir]

Huge regression on Dragon quest Builders (BJLM61311), i told you that already but it's official now. :)

Many objects, items, trees, etc.. are invisible now, They are present but invisible.

Game is unplayable now. Performance are good however.

RPCS3.zip

On master the game is perfect.

[Xcedf]

Played Dante's Inferno for two levels flawless 60 fps on my 6700k, other games are speeded up too another great improvement

[Xcedf]

Some games like Bionic Commando or Heavy Rain which had horrible frame-drops on master builds after having shader cache running pretty much smooth, another improvement RDR no longer seems to have random geomtery artifacts

[Luffykun007]

@kd-11 I'm having this mistake now in Ni No Kuni [NPEB90452]

E {rsx::thread} HLE: Getting the real address failed. (addr=0x1da3900) E {rsx::thread} RSX: Invalid FIFO queue get/put registers found, get=0x1DA3900, put=0x41FD40

[SaltyCucumber]

Don't know if placebo or not, but some parts of Demon's souls runs smoother. Though there was once a strange glitch where all the ground textures were dark green, I warped back and forth from nexus and it was gone. After more running around it finally froze during loading. I'll add the log just in case. RPCS3.zip

[kamer1337]

@kd-11 game stutters every first time visit the spot then it run smooth but it happens every run

[MarcoEstevez]

Firstable, congrats, you @kd-11 did an amazing job, I tested just 2 or 3 weeks ago Rune Factory Oceans and I was lucky if It reached 7-12 fps on the fist intro-tutorial battle, few days later with another commit, game almost reached 20-25 fps, enough for enjoying this low pacing game, but with current commit its playing full speed all the time, seriously unbeliebable.

hats off to you :-D

[kamer1337]

@bobthecooldad if it doesnt crash or anything just stuck then you have to wait minute or more for the first time run

[Illynir]

@Luffykun007 Same here, but only with Vulkan. On OGL it's good.

[JusicP]

@Luffykun007 Same thing with Battlefield 4 on OpenGL and Vulkan BF4 VK.zip BF4 GL.zip

[Luffykun007]

@illynir The same problem with the rsx reported above with the Vulkan api also happens in Opengl!

[Illynir]

Oh, ok. :(

[kd-11]

If you were getting stutter every time even after first run, please retest. @SaltyCucumber DeS has had that issue for a long time. You can watch lobosjr's emulator run and you'll see the same bug. I've experienced more drastic versions of it including the whole character model being green or red

[kamer1337]

latest commit fixed stuttering

[kd-11]

@Illynir I need the opengl renderdoc trace showing before and after

[kamer1337]

Unreal Engine 3 games like to crash the emulator on this build with vulkan

tony hawk pro skater hd and tony hawk pro skater 5

[kamer1337]

broken config in 31a9988 cant open, crash emu

[kamer1337]

it uses 100% of my ram and goes up. i play for a while and there is no more free ram

memory leak? or does it use huge amount of ram depending on how much cached shaders are

[Illynir]

@kamer1337 You have a nvidia card ? If yes, it's a bug of high consumption ram with the driver Nvidia. CEMU has the same problem.

Globally i think that this shader cache will have the same problem with high consumption of ram depends of quantity of shaders.

Problematic.

[kamer1337]

yes, i have nvidia

[Illynir]

I thought so... So no solution for that, it's a Nvidia driver bug. :-/

[tabnk]

Tested Virtua Fighter 5 Final Slowdown. Amazing huge speed up from 61 to 120 fps. :) Well done.

[Asinin3]

~~Ni No Kuni completely freezes (not crash) for me in 0.0.3-5656 with Vulkan after 1-20seconds, although I do seem to be the odd one out because others seem to be able to play it just fine. RPCS3_Ni_No_Kuni_Vulkan_Freeze2.zip~~

~~Last two lines when it freezes: F {rsx::thread} class std::runtime_error thrown: Unknown wrap mode 0xf (in file C:\rpcs3\rpcs3\Emu\RSX\gcm_enums.cpp:588)~~

~~And again (after running it again): E {rsx::thread} RSX: Invalid RSX method 0x2004 (arg=0x0) (in file C:\rpcs3\rpcs3\Emu\RSX\rsx_methods.cpp:45)~~

~~I am using windows 10 (so it shouldn't be related to the win7 hang in DeS) CPU: R7 1700 OCed @ 3.9ghz GPU: GTX 1080ti~~ This issue is not caused by this PR.

[Dokman]

@zminhquanz This is a high performance update, and it requieres a good graphics card

[Redfry]

@Asinin3 yes, i have this problem too, but on OGL Ni no Kuni works very good stable 20fps sometimes drop to 19 and up to 24 in start location. I am using win7 CPU I5 4460 GPU R7 250X

[OverlordCW]

RDR GOTY [BLES01294] F {rsx::thread} class std::runtime_error thrown: Unknown magnify filter 0x0 (in file C:\rpcs3\rpcs3\Emu\RSX\gcm_enums.cpp:631)

F {rsx::thread} class std::runtime_error thrown: Working buffer not big enough, buffer_length=268435456 allocated=266424320 requested=2097152 guard=65536 largest_pool=3140096 (in file c:\rpcs3\rpcs3\emu\rsx\d3d12../Common/ring_buffer_helper.h:74)

F {rsx::thread} class std::runtime_error thrown: Unknown stencil op 0x8180 (in file C:\rpcs3\rpcs3\Emu\RSX\gcm_enums.cpp:705)

RPCS3.zip

[Redfry]

@Asinin3 fps drop to 15 and freezes 1-2 times every 1-2 sec

[Asinin3]

@Redfry I'm not talking about stutter for a few seconds, that is completely normal. This build generates a shader cache now and that's why it's stuttering. If you let vulkan run for awhile the stutter will disappear once it has finished generating the cache. I was talking about a complete and total freeze, the window stays open and the audio keeps playing but the game remains frozen with an error in the log.

[Redfry]

I'm testing build's in start location and on this build total freezes gone, but maybe they can happen in another location. Now, in start location, i can play without any errors and total freezes

[Asinin3]

@Redfry Please read the comment I posted the stutters are completely normal until the cache is generated. Then delete your posts to keep this clean and not add any confusion/waste the developers time.

[kd-11]

Freezing with corrupt heap might be just from increased framerate. Test on master with null renderer and it might still happen. Bugs introduced by this pr would be driver crashes amd...dll or nvoglv..dll. That or missing/corrupt graphics. No part of this pr changes writes to emulated host memory.

[Asinin3]

@kd-11 Confirmed, just tried with Null renderer and when I loaded save it froze within 20seconds with the same error. It doesn't happen with OpenGL though, only Null and Vulkan.

[kamer1337]

is it possible to include disable shaders cache option? due to nvidia bug and high ram usage.

[Dokman]

@kamer1337 w8 until nvidia fix it

[SmileX-san]

@kd-11 Please include an option to disable shader cache system, because that stuttering on the 1st run (also followed by video/audio desync) spoils the whole gameplay. I tested Catherine. On master build it runs flawlessly on my configuration (Core i7-6700K + GTX 1070), but on this PR shader cache stuttering is very annoying. It happens everytime I enter a new location or a new cutscene and makes shader cache system absolutely unwanted for my configuration. That's why I'm asking to include an option to disable shader cache system.

[Dokman]

@SmileX-san w8 until nvidia want to fix the ram bug or use more ram

[SmileX-san]

@kd-11 @Dokman RAM usage is not an issue for me. I have 32 GB of RAM. The real issue is the increased complexity of the shaders in this PR followed by a shader cache system implementation. It is a probable solution for some games which run slow at the moment. But it also spoils the games that ran flawlessly before.

[SakataGintokiYT]

@kd-11 @Illynir

[Dragon Quest Builders ] Master-RenderDoc: Vertex_rewrite-RenderDoc:

[Zangetsu38]

@SakataGintokiYT i have already send this directly ^^

[Illynir]

@SakataGintokiYT Thx. ;)

[tabnk]

Speed improved greatly for Outrun online arcade. :).

[kd-11]

Dragon Quest Builders should be fixed now hopefully.

[Illynir]

I will test now. ;)

[Illynir]

Sorry KD but it's worse now. Flickering anywhere AND invisible object.

[kd-11]

Last commit was buggy. Retest.

[Luffykun007]

Oh shit! however Performance improvement in several games,thanks KD-11 nice work.

[SakataGintokiYT]

@kd-11 Fixed

[Illynir]

Yes it's fixed.