Texture scaling does not get applied on all textures

[VIRGINKLM]

I had to put an issue on that weird bug. I have set texture filtering on XBRz x5 just to be sure it's not my imagination. I noticed on Sword Art Online - Infinity Moment, the characters on the speech sequences only their first texture gets filtered and the rest are left in nearest neighbor. Take a look at this:

1st frame, texture gets properly filtered:

A couple of frames later from above frame, texture gets reset to nearest neighbor:

[unknownbrackets]

When a texture is modified, it doesn't get scaled ever again. Maybe we can do it based on # of frames between modification or something. Otherwise it will cripple performance in some games (even if most textures would be scaled fine.)

-[Unknown]

[VIRGINKLM]

Guessed so, but I haven't encountered it on other games though that do exactly the same thing with the amount of textures get loaded, let's say, Shining Ark so I had no idea why it would happen just on this game.

[PeterTh]

This does happen on some textures in some other games (e.g. Corpse Party) as well. As unknown says, we need to improve the heuristic which decides when a texture should be scaled.

Another option would be to add a user setting to disable the heuristic, but I wouldn't want to add more options if the problem is solvable adequately in the general case by improving the heuristic.

[daniel229]

Still not fixed yet.after blink,the xbrz effect gone.

[unknownbrackets]

Does this still happen? I thought we made it regain trust with the max texels thing.

-[Unknown]

[daniel229]

Still happening,like when eyes blicking,the scaling effect will be gone.

[unknownbrackets]

Does it eventually come back, or is it gone permanently?

If the blink is animated, we probably can't reasonably scale the blink, especially if the texture is large... hmm, well, maybe if it's the only thing changing on screen.

-[Unknown]

[daniel229]

It won't come back.

[unknownbrackets]

If you remove these lines, it works right? How is the performance?

                if (entry->numFrames < TEXCACHE_FRAME_CHANGE_FREQUENT) {
                    entry->status |= TexCacheEntry::STATUS_CHANGE_FREQUENT;
                }

Maybe we should just rely entirely on texels per frame.

-[Unknown]

[daniel229]

Yes,it works.Some game may get much low performance,some games doesn't. off

2x

off

2x

[unknownbrackets]

What if, instead, you decrease this number (in TextureCache.h) to something more like 60?

const static int FRAMES_REGAIN_TRUST = 1000;

-[Unknown]

[daniel229]

60 won't work,it work a moment when blicking then disappear.

[Kolrath]

I've notcied this occuring in numerous 2D games when using filtering. I had always chalked it up as being a glitch related to PPSSPP as I've not noticed it to occur in 2D games on other system emulators such as the PS2 emulator PCSX2.

From what I'm gathering from reading in this thread, as well as issue #2840 and #4235 mentioned above, is that this is the result of PPSSPP not filtering things that are regularly being updated as it would impact performance. Is this understanding correct?

If I am understanding that correctly then I've got a further question of... what is going on with the game Astonishia Story in relation to filtering?

When using XBRz, regardless of whether its 2x, 5x, auto, or whatever, it seems that only the animated objects are having the scaling applied while any static objects are left being pixelated.

In the example image below the two character sprites are being filtered as is the clock and the two barrels with water in them. These are all objects that are animated and constantly changing. However, the static objects like.... well everything else actually... are left unfiltered.

Would I be right in assuming this isn't the intended function?

I tried messing around with countless combinations of the various filtering and scaling options to see if anything had any different results but in all cases the animated stuff was being filtered and the static stuff was not.

I'm noticed sporadic / odd filtering happening in numerous other 2D games and from the standpoint of casual observation it seems fairly random. For example, in the game Grownlanser IV - Wayfarer of time some objects and characters would be filtered and others wouldn't. It seemed random in that sometimes the players moving character would or wouldn't be filtered or sometimes just a couple static objects or characters wouldn't be filtered or when two people are attacking each other and both are animated only one of them would be being filtered, etc.

[LunaMoo]

^ You mix filtering with scaling, but I guess you mean the latter one all the way. PPSSPP has limits on texture scaling for performance, both in frequency the texture changes and on amount of work that it can do per frame. Without them, some games can easily fill a few gb's of video ram and put constant abuse on cpu, in other words lag like crazy even on very powerful modern desktops.

However your example doesn't really show anything wrong with how those limits work ~ each game is different and especially 2D games are often working in very weird ways. This game looking at the screenshot for example could be using cpu to generate background graphics as a single texture starting from small tiles and that resulting texture would be slightly different every frame or two either by some effect or even by some trash data from game memory it copies the texture to/from. Even unnoticeable difference matters as it really must be pixel perfect to be threated as same texture and so not ignored by texture scaling. Animated objects in this one more than likely keep all frames of animation as one simple texture per object or maybe even whole page of them that never changes that's why scaling works on it.

You can just use GE debugger(on windows at least) ctrl+G or inside debug menu to observe step by step how graphics is made and see for yourself how the background is made. The function certainly does work as intended, the result in some games might be far from ideal through, that's why an issue like this exists in the first place.

[unknownbrackets]

They may seem static, but they may actually not be. For example, many 2D games:

A. Have animated sprites, e.g. images of people or animals that move around. A1. They may change these images for each animation frame. IIRC, Tales of Eternia does this. A2. They may keep separate frames all available as separate textures, allowing animation to use scaling and long-term cache.

B. Have background graphics, as a tilesheet. B1. This may be entirely static, which should be cacheable. B2. This may only draw static parts of the texture, in an "L" shape. Other areas of the texture may get updated for temporary usage, just because it's "free" memory to squirrel away data into. This counts as a change to the texture, even if that changed part is never rendered. B3. This may be animated. Parts of the tilesheet may be updated to account for e.g. rippling water or fireplaces. Whether drawn or not drawn, these count as changes to the texture. Popolocrois does this, for example.

So just because it uses a friendly method for the sprites doesn't mean it does for the tiles. Some of the above is just as @LunaMoo mentioned, I'm just clarifying why it's not necessary that both parts will behave the same way.

-[Unknown]

[Kolrath]

Thank you both for the information. All this kinda stuff is still a good bit "alien" to me so I don't really fully grasp how everything works. Not overly surprised that I was mixing up my usage of filtering and scaling.

This is likely another stupid question here, but I'm curious...

Instead of address each individual sprite / texture, would it be feasible to achieve similar result by just taking the entire image / surface before its rendered to the screen and modifying it? I was always under the impression that post-processing effects were less demanding since they only need to be done once to the "finished" image before its displayed on screen. I'm aware that wouldn't exactly be the same thing as the currently filtering / scaling methods, but it seems like it could result in the kinds of image quality that people are expecting to see based on their example screenshots in this thread.

Sorry if its a dumb question. Just curious about it.

[LunaMoo]

Yeah well we have a separate issue opened for that #6280 There are some postprocessing shader versions of xBR and other upscaling shaders which for fully 2D games could possibly work better than texture scaling, the thing is nobody made any good one for PPSSPP as of yet.

Some other emulators do have it and I belive libretro as well which do have ppsspp core(?) so maybe you could use it to get a functional full screen upscaling postprocessing shader on ppsspp. Never tried using libretro through, so not really sure about that. I did tried to port some existing xBR postprocessing shaders for ppsspp in the past, but maybe I'm just too stupid to understand how the scaling should happen through postprocessing as I was only able to get xBR rounding still in low res.;p

PPSSPP does have one upscaling shader ~ Spline36 upscaler, you can try it - if I'm not messing that up technically it should upscale when you use low rendering res(x1) and high window size/fullscreen, in practice, well just try it. As for me it kind of just shifts pixels which I guess would be a simple sharpen method if the shift was more subtle and probably it would be if it was done after scaling, so I wonder if that shader doesn't have same problem as my failed tries to make upscaling shader and does not scale at all heh. Then again Henrik made that one, so it might work fine and just look ugly in psp games.

[hrydgard]

I didn't write Spline36, it was contributed. Either way it would indeed be nice with more post shaders. It's possible there are some issues with how scaling works, but it should be possible to get some xBR type shader working. If not there's a bug.

[LunaMoo]

Oh sorry, maybe I just remembered you doing some changes to it when I first saw it:].

I tried again today, this time I found 5xBR v3.5a already in GLSL, so there's even less to do(and potentially break) than porting from .cg/hlsl which I tried in the past Edit: nvm blackscreen, I made a stupid typo/fixed;p.

I used version posted by "guest"(that's his actual nickname, not some random person;p) near the bottom in http://www.razyboard.com/system/morethread-xbr-shader-pete_bernert-266904-6143523-0.html

pretty much the difference from PPSSPP would be those OGL2Size and OGL2InvSize, which I found mean: OGL2InvSize.xy: 1/texture_size.xy OGL2Size.x = texture x size (for example "2048" in "high internal resolution" setting) OGL2Size.y = texture y size (for example "1024" in "high internal resolution" setting)

which I belive would in PPSSPP translate to: OGL2InvSize.xy = u_pixelDelta.xy; OGL2Size.xy = 1.0 / u_pixelDelta.xy; u_pixelDelta is pretty much equal to pixel size on the screen with OutputResolution=True, or I messed that up?

Edit: Either the shader made by "Guest" had a bug, or what I found about OGL size and inv size was incorrect, but I managed to fix it for PPSSPP: Try it: http://www.mediafire.com/download/y155wo45ctff91k/5xBRa.7z Screenshot compare: http://screenshotcomparison.com/comparison/144920 Edit: added variant b, might be a bit slower, but possibly deals better with tiny details, check the text at the bottom: http://screenshotcomparison.com/comparison/144938 Hopefully last edit ~ just to see if I can port the original shaders from .cg now I added v3.7c + CRT-caligari(squared) and v4.0 noblend, I'm kind of lazy to post screenshots - no blend is similar to previous two just no blend;p, and CRT, well it has scanlines. Also I don't think higher version = better in here, v3.5a seems twice as fast if I look correctly at pixels per clock with amd gpu shader analyzer(couldn't really feel the performance difference in game even with unlocked framerate). (extract to assets/shaders directory). Also rendering res has to be set to 1, window size bigger/fullscreen and screen scaling filter set to nearest. @hrydgard guess extra shader would not be a big deal to add to incoming release?:P

Quick edit I just messed with that spline36 a bit and it doesn't seem to use outputresolution for anything with it's algorithm, it looks better with the awful artifacts changing into somewhat working sharpening effect when both "u_texelDelta" are replaced by "u_pixelDelta", however still kind of ugly with most of the pixels unchanged;p. I belive spline 36 should generate at least some blur and this is free of it either way. Maybe because it uses int for coordinates does that mean 1 = 1 pixel in rendering or output resolution? With the latter I suppose it would really not do any scaling, but I'm just learning, and that's pretty much first I see integers in GLSL/gotta read more;p.

Oh and also ~ sorry for that offtop;p, that really should all be in the other issue.

[daniel229]

Comment out this line not slow down the game now,but slow down the video,stop apply to video then may have a try.

                if (entry->numFrames < TEXCACHE_FRAME_CHANGE_FREQUENT) {
                    entry->status |= TexCacheEntry::STATUS_CHANGE_FREQUENT;
                }

[daniel229]

Some scene may still slow much,and still some scenes not get the effect.

[unknownbrackets]

How is it if you increase TEXCACHE_FRAME_CHANGE_FREQUENT from 6 to maybe 18?

-[Unknown]

[daniel229]

increase TEXCACHE_FRAME_CHANGE_FREQUENT from 6 to 18 didn't work.

[unknownbrackets]

Is it any better in the latest git build? Note that some textures may not immediately scale, mainly due to performance concerns. But I made it so they eventually scale much sooner, if possible.

@LunaMoo if you're still working on that shader, we could probably use it in two areas:

1. When games upload RAM to framebuffers via block transfers, we could scale using that shader.
2. As a post-shader.

The first option might be interesting for cases where the game downloads, and then re-uploads an area, to do effects on it. It might work better in games where the post-shader is not always desirable. Just an idea.

-[Unknown]

[LunaMoo]

Well 5xBR is already included in PPSSPP as post shader: https://github.com/hrydgard/ppsspp/blob/master/assets/shaders/5xBR.fsh Downsides of it are kind of obvious - certainly low res for 3D stuff and it also doesn't look as good as texture scaling with "pixel art" or any other stretched low res textures since using rendering res for pixel size.

That leaves 1 - would certainly not hurt to have an option like that at least some games could benefit from it, I have no idea how to do that through.

[unknownbrackets]

Oh, I thought xBR was different from xBRZ and misread your comment as being about xBRZ.

For 1 - that's DrawPixels. Every case that DrawPixels is called now is when rendering RAM into a framebuffer or the screen. Right now it always uses draw2dprogram_ for that. So the change would be making it use a different program, one probably initialized at the same time as the postshader (but probably a separate setting from the postshader.)

-[Unknown]

[unknownbrackets]

How is this looking now? We could implement something to skip scaling videos, although I worry that doesn't solve the general problem...

To some extent, texture replacement can now be used to work around this problem. Not calling that a fix exactly, but pre-scaled texture loading will perform better so doesn't need the same limits.

-[Unknown]

[daniel229]

The effect goes away as soon as it blinks,then comes back after 0.5 second.

[ghost]

Hello, is this normal if this is still happening with 1.2.2?

[unknownbrackets]

Yes, old releases are not retroactively modified. 1.2.2 isn't going to get changes made after it was released.

Also, this issue is still marked open, so it would of course be normal to still see it.

In either case, texture scaling is very expensive (especially at high scaling levels), and some games change textures every frame. This issue is really about scaling as much as is realistically possible - so if you're waiting for texture scaling in a game like FF2, you might need to get a 4.5Ghz 48-core CPU first.

-[Unknown]

[ghost]

Nah, I came here from the Disgaea (1 & 2) issue.

[Squall-Leonhart]

noticing in current buildbot builds that texture scaling will cause a stutter and skip during magic casts in Final Fantasy / Final Fantasy II

[ghost]

Auto Max Quality in texture scaling could help this issue?