(Regression) Gran Turismo 4 increased memory leak - crash with >2x scaling

[bositman]

-PCSX2 version: 1.3.0 GIT 425-g2cbde89

-PCSX2 options: Defaults

-Plugins used: GSdx, SPU2-X, Lilypad, Linuz ISO/Gigaherz CDVD,rest null plugins

-Plugin settings: Defaults

-Description:When entering a race, game uses all 2 GB VRAM of my graphics card then all 6 GB system RAM leading to a crash of..everything :P This occurs in both DX11 and OpenGL HW backends, SW works fine.

Note: The game already memory leaks (or uses insane amounts of memory...) but after the noted build it got much much worse

On native the game uses about 550 mb VRAM On 2x scaling, it uses all my 2 GB VRAM, about 5 GB free system RAM, crashes PCSX2, firefox, windows and everything else after running me out of memory :P In comparison, last known build to work reaches 1,7 GB VRAM and PCSX2 runs stable after the spike

This build works ok: https://github.com/PCSX2/pcsx2/commit/d793ef85df2c7111aae5559e7a9a6600ea3abd86 (still uses all my VRAM but does not leak any more, so game is playable)

This one and after are broken: https://github.com/PCSX2/pcsx2/commit/2cbde89084f1f7385d09f43e9ff96424bf35cb2a

So I suppose the game really hates the real-fb-format branch (although I can't test to make sure since there was no build for this: https://github.com/PCSX2/pcsx2/commit/f81cf360bce91649a5827967dc5e73c926711611 )

-How to replicate: Boot the game, enter a race

-Last known version to work: https://github.com/PCSX2/pcsx2/commit/d793ef85df2c7111aae5559e7a9a6600ea3abd86

-PC specifications: CPU: Intel Core i5 2500K @ 4.3Ghz GPU: Gigabyte Geforce GTX660 Windforce OC 2 GB OS: Windows 7 SP1

GS dump: http://bositman.pcsx2.net/dumps/gran-turismo-4-memory-leak-regression.rar

[gregory38]

Strange. Do you have any accutate option on? Allow 8 bit textures?

[bositman]

Allow 8 bit textures is off and it happens with both DX11 and OpenGL so I guess accurate options aren't the issue here. I just retested OpenGL with all accurate = 0 ,it was a bit better (only crashed firefox :P ) but it had other rendering issues ingame and still lots more VRAM/RAM usage than the old builds

[gregory38]

It might worth to check the memory consumption with 8 bits texture on.

I'm glad you mention f81cf360bce91649a5827967dc5e73c926711611 I think the issue is likely @refractionpcsx2 code. The size of the framebuffer is detected automatically. I guess the size is now bigger => more (V)RAM usage.

The real-fb-format only change the computing of alpha/color in shader. Virtually no impact on mem.

[gregory38]

How fast the memory is filled?

[refractionpcsx2]

GT4 has always been a memory hog, no card i've own has managed to run the game well above 2x native, ever :P The transitions are even worse than the rest of the game, they seem to really kill the card.

[bositman]

About 5 seconds? It spikes on a certain scene where all race cars are displayed, then goes down again, then dies before you take control of the car (since that view has all cars in it at close range) @refractionpcsx2 Yep, but I could do 3x native with the previous build, now I can only do native without crashing I'll try with 8 bit texture on

[refractionpcsx2]

Unless the game is using SS methods like Baldurs gate, the buffer size shouldn't be any bigger than it was previous to the snowblind changes.

Edit: Okay it might be a bit bigger, i lied :P I could shrink the default size for the buffer as it doesn't need to be as big as it is by default. if i make it 640*512 (kinda like it was originally) that should cover most games. for initialization.

[gregory38]

I was very close to say that I agree with you. But you change the size of heigh too (it is square now).

If I'm right, it confirms that target is the memory hog.

[refractionpcsx2]

Yeah, I did that as it's difficult to guess the height, was just being safe in case it caused any height issues, but I guess it seemed to work before.

[gregory38]

Yes, I understood it. And I don't think it was bad, 16 bit format will use a 2x bigger height.

I guess it also impact the slow down in Snow Engine Game. If the target is too big, we need to transfer more pixels for cache invalidation.

I propose to 1/ resuse the old way to compute the height 2/ reduce the lifespan of target to 10 frames instead of 400 frames (only to test the impact on memory consumption for this game)

[refractionpcsx2]

hmm, so it's best to keep it square for 16bit stuff, so i i will make the default 640x640, that should be fine :)

[gregory38]

No don't bother with 16 bits stuff (actually it is only 32bits target read as 16 bits target). I don't think it worth to emulate it, and it will requires differents code path anyway. Old code was likely the best for the height. It is really too costly. I think it might help also the baldur gate case which use a 16 bits targets too.

[gregory38]

As a side note, I'm curious of what would be the impact to use a real 16 bits RT on GL/DX like the PS2 instead of a 32 bits format. It mights reduce by 2 the memory requirements for those games.

[refractionpcsx2]

It would be interesting to try and possibly avoid a ton of bugs created from us converting it :)

[gregory38]

possibly avoid a ton of bugs created from us converting it :)

Why do you mean by it?

Currently the output of the blending is 32 unit and it isn't converted (except in accurate* stuff). I think it would fix a couple of bug but it might introduce others bugs (dithering?)

Or do you mean the conversion to the 16 bits GS the format? Or others?

[refractionpcsx2]

What i meant was if the PS2 frame buffer is 16 bits and we are doing it on a 32bit render target, surely we are translating stuff to 32bits, but i recall a bunch of these stripey screen bugs were caused by it being 32bits, ZeroGS had a "use 16bit render target option" which fixed a load of these issues.

So my point was that using 16bits as a point of accurate emulation would potentially fix a bunch of other bugs we may not know the cause of.

[gregory38]

16bit target must be doable. However stripey is typically 32 bits buffer read/written as a 16 bits which is a bit different. 16 bits target will allow to have a more accurate blend.

@bositman I partially reverted the buffer size. Could you tell me if it is better for the memory? And do you know how to compile PCSX2?

[bositman]

Yeah I can compile, I'll give it a try now

[bositman]

OK I can do 2x native now without crashing, 3x still crashes (borderline, but that was the same as before). I tried 8 bit textures and it did help (i could run 3x native relatively well) but it made the whole display flash on and off :P The game has some weird VRAM usage spikes, I'll make a video so you guys can check it. I guess this can be closed now, good job to both ;)

[bositman]

https://www.youtube.com/watch?v=G7RkSxfRA7M

Here's the video. You'll notice there are some huge spikes (which is when everything would crash at 3x native) it seems to be a fade effect? Posting this for another reason too, if you check 1:47 vs 3:20 you'll see a huge difference between OpenGL and DX11 at that blur effect. I believe this is the same issue as one I found at Fatal Frame 3, which is also game breaking and OpenGL exclusive. Will make a report of that later.

[refractionpcsx2]

Yeah the blur effects are a little broken on OpenGL atm with GT4. Latest commit Gregory did broke Baldurs Gate so im just fixing that now and finding a better way to determine heights etc

[refractionpcsx2]

Should be okay again now

[karasuhebi]

@bositman I also find it interesting that when your VRAM usage spikes, so does your pagefile usage, even though the RAM isn't being maxed out. Is that normal?

[refractionpcsx2]

Windows attempts to keep some ram free to stop your machine being completely crippled, so it bungs everything it can in high usage situations in the pagefile :P