Block transfer effects speed hack (fix)

[milansimek]

Hi,

I've modified some of the simulate block transfer effects code to allow it to only execute block transfer every xx frames.

For example, I've set it to every 3 frames when playing Ys Seven, since mainly the minimap needs block transfer effects to be enabled. Using the speedhack, the minimap is only updated 3x per second, but that's no issue of course. The game now runs smoothly at 30 fps on 4x PSP resolution, as compared to 25-30fps on 2x PSP resolution without the speed hack.

I'd be happy to create a pull request but I'm wondering where I should place the configuration for this speed hack? Should I add a checkbox / frameskip count underneath the "Simulate block transfer effects" checkbox, or somewhere else in the configuration screen?

Let me know, thanks!

[unknownbrackets]

Hmm, I think there are several things to consider here:

1. This could cause flicker in games that reuse RAM/VRAM for other purposes mid-frame, which might just cause confusion when the speedhack causes weird issues. This is common with shadows.

2. Some games will download data just one-off, for example a screenshot for a save icon, or an image of an object (rendered text, a billboard, a shadow, your created katamari, etc.) If this accidentally falls on the 2/3 frame, it would be bad. Maybe it needs to track 3-step per dest address with a timeout?

3. It seems a bit odd that this would cause smooth FPS. In theory, it would mean frame 1 takes 33ms, frame 2 33ms, then frame 3 40ms (instead of all 40ms before.) So it should still mean dropping a frame once every 3. How does it interact with frameskip and auto frameskip?

4. Does enabling #11531 for this game instead help? In theory, it might allow real time rendering of the minimap and full speed, best of both worlds. However, it can also mean that things like save icons break.

-[Unknown]

[milansimek]

I just checked my code and now see I've set the speed hack to do the block transfer every 5 frames, so 6x per second.

Below some comments / answers to the valid points you mention:

1. There is indeed some flicker when special moves are executed in Ys Seven. This however is less worse than the huge framedrop you normally get when executing these.

2. I can confirm this. I tried a couple of methods and managed to fix most issues regarding that (at least for Ys Seven). Still some times the image used for save games is black indeed, which is caused by the speed hack. When disabling block transfers completely for this game, some images get "stuck" in memory and will display when switching location in game. I found a place to put the frameskip code so that most stuff in game is rendered correctly, with virtually no cached images being displayed from the buffer when for example exiting / entering villages.

3. I haven't tested it with auto-frameskip yet since I've disabled that for the game. I'll check that and let you know. The thing is that with the regular block transfer effects enabled, the frame rate will drop constantly whenever there is a battle or when special moves are executed. Even when using only 2x PSP resolution. Your point regarding the frame rendering is valid, but to be honest I don't notice it when I play and it's definitely much smoother with the speed hack in place (at 4x PSP resolution).

4. I will definitely check that and report back here with my findings

FYI: I added the frameskip code here: https://github.com/hrydgard/ppsspp/blob/master/GPU/Common/FramebufferCommon.cpp#L1672

There if (g_Config.bBlockTransferGPU && !srcBuffer->memoryUpdated) { becomes if (g_Config.bBlockTransferGPU && !srcBuffer->memoryUpdated && CanRenderBlockTransfer(srcBuffer)) {

Of course it's not a perfect solution, but when choosing between slow gameplay in low resolution vs high resolution with occasional flicker, the second option is definitely better at least for this specific game :)

[milansimek]

Is #11531 already merged into the development builds available on the PPSSPP website? If that's the case then it doesn't increase performance much for Ys Seven unfortunately.

Or should I create a custom build to test this?

Thanks!

[hrydgard]

@milansimek It's currently not enabled for Ys, the feature has a whitelist. You have to add an entry to compat.ini for that path - and I don't know for sure that it will help.

[milansimek]

@hrydgard The proposed fix seems to work fine, and so far much better than the speed hack I used previously 👍

I created the file PSP/SYSTEM/compat.ini and added the following contents:

[BlockTransferAllowCreateFB]
ULUS10551 = true # Ys Seven

Will play the game a bit more and will report here and on #11531 in case I encounter any issues. Like you mentioned save game images are just black without contents, but that's not really a big issue.

[hrydgard]

Glad to hear that. I think I'll merge it (and also add the other GameIDs that I can find for the game).

[unknownbrackets]

In that case we may want a way to turn it off - some people may have perfectly powerful desktops and prefer working save icons to not-any-faster emulation.

-[Unknown]

[hrydgard]

Right - are those reads in any way different from those during the runtime that cause slowdowns, or are they the same?

[unknownbrackets]

In one area at least, every frame it downloads 04126000 -> 08c86100, which seems to be a minimap. See #2928. This is actually a small framebuf. This is even downloaded when the "minimap" option is set to "No display", unfortunately. AFAICT, this can safely use a fake FB.

When showing the full map (so not usually), it renders the scene from scratch, then downloads that (04044000 -> 08c3bfc0), then clears & renders that back to the framebuffer with vertex colors, then draws the map on top. This could also safely use a fake FB.

When you enter the menu (not every frame), it downloads 04044000 -> 08c3bfc0 (so same as full map), but only once. This is used for the save screenshot (if you save; it unconditionally downloads it when entering the menu in case you choose to save.)

Notably, this save icon related download is done by the same function as other downloads, so isn't easy to tell apart.

In another scene with more lava and effects, it downloads 04044000 -> 08c3bfc0 every frame. This is done right after the minimap download. It's used to show a "heat" effect with things shifting around a bit as you see things through refracted smoke and flames. This seems safe to be a fake FB too.

At least (unlike some other games...) it's mostly not doing readbacks pointlessly. But I'm not sure how to detect just the savedata one. There might be more effects, of course.

-[Unknown]

[hrydgard]

Hm, tricky. One possibility (thinking a bit far out here) would be to not to the transfer but mark the pages as inaccessible, and as soon as a read hits, actually schedule a readback. Though that of course could result in the readback happening too late... and requires that read/write trap infrastructure that PPSSPP doesn't yet have (but we could just grab from Dolphin).

[unknownbrackets]

Right, I wonder if that works on toasters though.... maybe it does. We could sync the actual data in the trap, perhaps...

It's likely that some code reads from 08c3bfc0, so perhaps a hook could be added that downloads any RAM framebuf that's not materialized at the src ptr... obviously not a very generic solution, though.

-[Unknown]

[Spicy-Kimchi]

Tested on Naruto Shippuden Ultimate Ninja Impact and work perfectly fine,wonder when will this get merge

[hrydgard]

@C4rrotMilk Does it help performance a lot in that game?

[ghost]

It's super lag if "block tranfer effect" is enable in Naruto Shippuden Ultimate Ninja Impact but it's fixed the small black square on the screen

[Spicy-Kimchi]

It helped a lot, without it the fps would drop below 10 on my phone(Huawei Mate 9). But it will cause the video flickering like having double image, not really a big deal tho since the video still play smoothly

[ghost]

Well your phone is a mid-ranger or high-end what about others like me that have a low specs phone and really doesn't gain any FPS increase from "block transfer effects"? but instead decrease the performance by 50% 🤔

[Saramagrean]

@Emulatorer He means in the game Ys Seven, When enabled "BlockTransferAllowCreateFB" (in compat.ini file not settings) is improve performance a lot.

I tested with Naruto Shippuuden Impact is help too but a little.

[hrydgard]

@Emulatorer Block transfer effect turns off some of the PSP's graphics functionality, so unchecking it can speed things up (and cause problems), not checking it. This new per-game setting BlockTransferAllowCreateFB avoids some situations where block transfers could cause unreasonably bad performance, but doesn't work in all siutations.

[Spicy-Kimchi]

Here a comparison with it on/off

Off: https://imgur.com/U8DmPUE

On: https://imgur.com/oc3Vg98

Should be frame overlap i think, sry my English not that good

[ghost]

Sir @hrydgard it's ok now to add Naruto Shippuden Ultimate Ninja Impact in BlockTransferAllowCreateFB compat.ini list :)

[ghost]

Game that probably should be added to Virtual Readback compat.ini before releasing v1.8.0 #11861

Naruto Shippuden Ultimate Ninja Impact -fixed small black square when block tranfer effect is on game performance still the same Marvel Ultimate Alliance 1 & 2 -game is running better now when block tranfer effect is on

[unknownbrackets]

In theory, it will make every game run faster. However, it MAY break effects.

For example, Hexyz Force uses block transfer. There is a specific point in the game where you get a scene where the graphics are supposed to be colored to make it not look like "reality." This won't happen with virtual readbacks and it will render incorrectly.

However, if you just started the game and got through some dialog and battles, you'd probably decide it had no problems with virtual readbacks, or that it only made the game faster. It does not do that effect everywhere.

Other effects, like specific special attacks or scenes, may also render wrong - sometimes badly and obviously, sometimes it will just look worse or make less sense, but you won't be sure if that's correct or not.

-[Unknown]

[ghost]

Burnout Legend also benefits the virtual readback Fixing the Sun black square issue also the game is running smooth even block tranfer effect is enable 🙂

[Panderner]

In Gradius collection it's uses block transfers used in all games except Gradius IV. When I starting the game when block transfer is off it result black screen and when I turn off while it's in-game the game freezes

[rayanfer32]

Block transfer is still makes the game laggy ...however if i disable it creates an overlay on naruto impact .... im struck it is making this awsome game unplayable.... plz add a setting to tweak this somehow..

[Panderner]

Test Drive Unlimited Requires Simulated Block Transfer Effects to be turned on to play an entire game but it's super slow

[ghost]

I think this issue is related to #12345

[Panderner]

I've enabled BlockTransferAllowCreateFB for Naruto Shippuden: Ultimate Ninja Impact. It greatly increases performance for low end devices.

[ghost]

I've enabled BlockTransferAllowCreateFB for Naruto Shippuden: Ultimate Ninja Impact. It greatly increases performance for low end devices.

Not really after you enable it by default Naruto Shippuden became very lag on my low specs android phone (Cloudfone Excite Prime)

[Panderner]

I've enabled BlockTransferAllowCreateFB for Naruto Shippuden: Ultimate Ninja Impact. It greatly increases performance for low end devices.

Not really after you enable it by default Naruto Shippuden became very lag on my low specs android phone (Cloudfone Excite Prime)

@Gamemulatorer can you compare the performance for BlockTransferAllowCreateFB change?

[hrydgard]

Just for future reference, ran into this reddit thread where some guy tries BlockTransferAllowCreateFB in a bunch of games:

https://www.reddit.com/r/EmulationOnAndroid/comments/be676t/incredible_wonder_of_ppsspppart2/

of course, can still be breakage in these.

[ghost]

Hot Shot Get a Grip and Everybody Tennis requires simulate block transfer effect for replay video correct graphics.

[ghost]

Gensou Suikoden need simulate block transfer effects for correct graphics.

simulate block transfer effects OFF

https://user-images.githubusercontent.com/37603562/137293826-e1a48fa8-66d8-417c-a8fc-8bc5976cd1a2.mp4

simulate block transfer effects ON

https://user-images.githubusercontent.com/37603562/137293717-e1e8232d-d914-413a-9525-4c9f7b0709a9.mp4

[ghost]

Enable Simulate Block Transfer Effects can causing graphics glitch. See #8594 and #6265.

[ghost]

MotorStorm Artic Edge and Silent Hill Shattered Memories required SBTE enable to fix the missing graphics in some scene but can cause huge fps drop.

[ghost]

This is still an issue?

[hrydgard]

We're not going to do this particular speedhack. For cases like this minimap issue, delayed readbacks seem more appropriate, so might enable those for this.