Kingdom Hearts: BBS pgd and texture problems

[c0nK3R]

Here's my post in the forum: http://forums.ppsspp.org/showthread.php?tid=445&pid=7797#pid7797

First PGD decrypting doesn't work on this game. A clean dump runs into a million of these "W[MM]: MemmapFunctions.cpp:91 ReadFromHardware: Invalid address 0000001c". Where a decrypted one can actually play.

Second I believe the emulator is destroying the textures prematurely for this game. Here's an example of a treasure chest monster, as it transforms it will display its textures for a millisecond before being wiped away. (Happens every time Which is perfect for debugging :)) here's a precise log from that event http://pastie.org/pastes/6483852/text (I have indicate when I've opened the box and when the box has already turned back to black by breaking the flow of the log)

There's more in the forum post so check that as well.

If you need more info or test post what you need me to do either here or in the forum. Looking forward to this game working :)

[tpunix]

PGD decrypting have problem. I'll try to fix it.

[gametaster]

looking forward to see the progress

[tpunix]

the PGD function should be OK now. you can try this game again.

[gametaster]

I still cant see the menu :(

[gametaster]

I have test the kingdom hearts: BBS and kingdom hearts:BBS final mix And it have same problem,, here the screenshot from my xperia ray

and after I open the box and want to move to next place,it force close,I have un check the fast mem and buffered rendering and it still force close Sent from BlackBerry® on 3

-----Original Message----- From: tpunix notifications@github.com Date: Sat, 30 Mar 2013 00:43:59 To: hrydgard/ppssppppsspp@noreply.github.com Reply-To: hrydgard/ppsspp reply@reply.github.com Cc: gametasterakbar_alchemyst@yahoo.co.id Subject: Re: [ppsspp] Kingdom Hearts: BBS pgd and texture problems (#979)

the PGD function should be OK now. you can try this game again.

---

Reply to this email directly or view it on GitHub: https://github.com/hrydgard/ppsspp/issues/979#issuecomment-15670987

[gametaster]

Oh I forgot,I test in latest build v0.7-349 Sent from BlackBerry® on 3

-----Original Message----- From: tpunix notifications@github.com Date: Sat, 30 Mar 2013 00:43:59 To: hrydgard/ppssppppsspp@noreply.github.com Reply-To: hrydgard/ppsspp reply@reply.github.com Cc: gametasterakbar_alchemyst@yahoo.co.id Subject: Re: [ppsspp] Kingdom Hearts: BBS pgd and texture problems (#979)

the PGD function should be OK now. you can try this game again.

---

Reply to this email directly or view it on GitHub: https://github.com/hrydgard/ppsspp/issues/979#issuecomment-15670987

[c0nK3R]

@tpunix PGD decryption works fine now. Sorry for not getting back to you sooner. @gametaster the menu is not part of the pgd error but something else associated most likely with the characters texture problem.

I have made a video, and hopefully it will provide some insight into what is happening with the textures. http://www.youtube.com/watch?v=CLhkjm1uTvw

@unknownbrackets This one might interest you. It deals with save state breaks -Last tested with v0.7-650-gffe88f5-windows-x86 http://www.youtube.com/watch?v=4Qp8m1wZlsU here is a log http://pastebin.com/raw.php?i=R8QnU8ta where it hits the "Invalid address" spot is where the game breaks for whatever reason.

[unknownbrackets]

PGD doesn't properly save state, so unfortunately that's expected right now.

-[Unknown]

[c0nK3R]

Alright awesome just pointing it out in case it wasn't known yet. I just thought it was weird that it breaks in the next room rather than breaking right away like previous save state bugs.

[dbz400]

This game drives me crazy to get in-game without menu , skip CG :) , finally get in-game .........

[dbz400]

Hack the fragment shader to get it show the menu and character .... sigh .... though wrong color ....

[unknownbrackets]

What part do you change to make it show things?

-[Unknown]

[dbz400]

I changed the WRITE(p, " vec4 p = v_color0;\n"); to WRITE(p, "vec4 p = texture2D(tex, v_texcoord);\n");

if (gstate.textureMapEnable & 1) { WRITE(p, " vec4 t = texture2D(tex, v_texcoord);\n"); WRITE(p, " vec4 p = v_color0;\n");

[dbz400]

This WRONG hacking makes lots of missing/black texture and special effect show , unfornaturely i didn't get a proper fix .....

[hrydgard]

well, that proves stuff is getting drawn, just need to figure out where the black is coming from :)

[unknownbrackets]

Maybe the real problem is in the vertex shader then? It looks betterish when a_color0 (which seems to always be transparent black) is ignored. It also looks betterish when matambientalpha is used instead of a_color0 (but still wrong.) But I think those are all the wrong fixes...

-[Unknown]

[cinaera]

@raven02: Could you post the shader codes? They should be displayed in the visual studio output window while debugging

[unknownbrackets]

@cinera, if I understood correctly he just changed:

WRITE(p, " vec4 t = texture2D(tex, v_texcoord);\n");
WRITE(p, " vec4 p = v_color0;\n");

To:

WRITE(p, " vec4 t = texture2D(tex, v_texcoord);\n");
WRITE(p, " vec4 p = texture2D(tex, v_texcoord);\n");

Basically, it's because v_color0 is vec4(0, 0, 0, 0), and it uses MODULATE with alpha:

WRITE(p, "  vec4 v = p * t%s;\n", secondary); break;

Which basically results in v = vec4(0, 0, 0, 0), of course. That's why I think the real problem is that v_color0 is wrong. Anyway, here are the shaders:

#version 110
#define lowp
#define mediump
attribute vec4 a_position;
attribute lowp vec4 a_color0;
uniform mat4 u_proj_through;
varying lowp vec4 v_color0;
void main() {
  v_color0 = a_color0;
  gl_Position = u_proj_through * vec4(a_position.xyz, 1.0);
}

#version 110
varying vec4 v_color0;
void main() {
  gl_FragColor = v_color0;
}

#version 110
#define lowp
#define mediump
attribute vec4 a_position;
attribute vec2 a_texcoord;
attribute lowp vec4 a_color0;
uniform mat4 u_proj_through;
varying lowp vec4 v_color0;
varying vec2 v_texcoord;
void main() {
  v_texcoord = a_texcoord;
  v_color0 = a_color0;
  gl_Position = u_proj_through * vec4(a_position.xyz, 1.0);
}

#version 110
uniform sampler2D tex;
uniform vec4 u_alphacolorref;
uniform vec4 u_colormask;
uniform vec3 u_texenv;
varying vec4 v_color0;
varying vec2 v_texcoord;
void main() {
  vec4 t = texture2D(tex, v_texcoord);
  vec4 p = v_color0;
  vec4 v = p * t;
  v = v * 2.0;
  if (v.a  <=  u_alphacolorref.a) discard;
  gl_FragColor = v;
}

#version 110
#define lowp
#define mediump
attribute vec4 a_position;
attribute vec2 a_texcoord;
attribute lowp vec4 a_color0;
uniform mat4 u_proj_through;
varying lowp vec4 v_color0;
varying vec2 v_texcoord;
void main() {
  v_texcoord = a_texcoord;
  v_color0 = a_color0;
  gl_Position = u_proj_through * vec4(a_position.xyz, 1.0);
}

I'll note that it's using rectangles, so it is doing software transform at this point (this is from the dialog at the very start.)

Edit: here's the frame dump too: https://gist.github.com/unknownbrackets/76133f815524ebc2d336

-[Unknown]

[dbz400]

I also think it is input v_color0 issue , here is the shader code which i commented out #define DEBUG_SHADER and have p set to equal to t which is texture2D(tex, v_texcoord)

-This is the battle which original have black texture character

version 110

uniform sampler2D tex; uniform vec4 u_alphacolorref; uniform vec4 u_colormask; uniform vec3 u_texenv; varying vec4 v_color0; varying vec2 v_texcoord; void main() { vec4 t = texture2D(tex, v_texcoord); vec4 p = texture2D(tex, v_texcoord); vec4 v = p * t; if (v.a <= u_alphacolorref.a) discard; gl_FragColor = v; gl_FragColor = texture2D(tex, v_texcoord); }

version 110

define lowp

define mediump

attribute vec3 a_position; attribute lowp vec4 a_color0; uniform mat4 u_proj; uniform lowp vec4 u_matambientalpha; uniform mat4 u_world; uniform mat4 u_view; uniform vec4 u_uvscaleoffset; varying lowp vec4 v_color0; void main() { vec3 worldpos = (u_world * vec4(a_position.xyz, 1.0)).xyz; vec4 viewPos = u_view * vec4(worldpos, 1.0); lowp vec3 unlitColor = a_color0.rgb; v_color0 = a_color0; gl_Position = u_proj * viewPos; }

version 110

uniform sampler2D tex; uniform vec3 u_texenv; varying vec4 v_color0; varying vec2 v_texcoord; void main() { vec4 t = texture2D(tex, v_texcoord); vec4 p = texture2D(tex, v_texcoord); vec4 v = vec4(mix(p.rgb, u_texenv.rgb, t.rgb), p.a * t.a); gl_FragColor = v; gl_FragColor = texture2D(tex, v_texcoord); }

[cinaera]

Thanks for posting. I agree could be something with the color (though can't find anything suspicious) or maybe there is something wrong with the alphatesting

[dbz400]

I also tried to disable alpha testing and seems not helping ...

[unknownbrackets]

Per:

http://forums.ppsspp.org/showthread.php?tid=445&pid=15774#pid15774

Is it possible that (at least some of) this is a clamping issue?

-[Unknown]

[unknownbrackets]

Some other notes / possibilities:

VFPU used (in case it's a CPU bug?): vuc2i, vmmul, vi2f, vtfm, vscl, vf2iz/vf2id, vi2uc, vmscl, vidt, vrot, vcst

Uses pos DEC_FLOAT_3, UV DEC_U16A_2, and color0 DEC_U8_4. The vertex decoder is only reading colors that have all components as 0.

Flushing between every command doesn't change anything.

So I guess the next question is, are the vertex colors correct? I'm not sure how to tell. If they're not, it's probably a cpu bug I guess?

-[Unknown]

[hrydgard]

Sending only zero vertex colors is highly suspicious, in that case the game wouldn't need to send any vertex colors at all. Smells like CPU bug indeed.

Of course, games do many silly things with graphics, setting states over and over for no reason at all for example, so you never know...

[unknownbrackets]

Another note, as far as textures, it's using 4-bit GE_CMODE_32BIT_ABGR8888 textures, and also swizzled (rowWidth >= 16) 8-bit indexed textures, also 32-bit clut. So that's pretty standard.

So yeah, the 0'd vertex colors are pretty darn suspicious.

-[Unknown]

[unknownbrackets]

Well, I'm out of time, but the first one I tried, vuc2i, made the text visible:

        switch ((op >> 16) & 3) {
        case 0:  // vuc2i  
            // Quad is the only option.
            // This operation is weird. This particular way of working matches hw but does not 
            // seem quite sane.
            {
                u32 value = s[0];
                u32 value2 = value / 2;
                for (int i = 0; i < 4; i++) {
                    //d[i] = (u32)((value & 0xFF) * 0x01010101) >> 1;
                    //value >>= 8;
                    d[i] = 0x88888888;
                }
                oz = V_Quad;
            }
            break;

Doesn't mean it's vuc2i but yeah, I'm going with vfpu bug.

-[Unknown]

[hrydgard]

yup, that seems pretty conclusive. I believe all those ops are pretty well tested though...

[dbz400]

I tested the above vuc2i on other games .See below

-This one is correct and same as real PSP

[dbz400]

Then i changed to this code for vuc2i

    case 0:  // vuc2i  
        // Quad is the only option.
        // This operation is weird. This particular way of working matches hw but does not 
        // seem quite sane.
        {
            d[0] = (((s[0]      ) & 0xFF) * 0x01010101) >> 1;
            d[1] = (((s[0] >> 8 ) & 0xFF) * 0x01010101) >> 1;
            d[2] = (((s[0] >> 16) & 0xFF) * 0x01010101) >> 1;
            d[3] = (((s[0] >> 24) & 0xFF) * 0x01010101) >> 1;
            oz = V_Quad;
        }
        break;

-This one definitely incorrect

-This one is same as real PSP

[dbz400]

However , two approachs will make character disappear in Saint Seiya Omega at the same time :(

[vontman]

Can't u make these games only use the method above?

[unknownbrackets]

Just to clarify, my change was ONLY to determine if VFPU was generating these color values. My code sample BROKE that vfpu instruction intentionally for the explicit purpose of identification.

Looking into it a tiny bit more a several hours ago, it was vmul.q'ing by 0, so I need to figure out where the initial 0's are coming from. The vuc2i instruction was getting 0 in and sending 0 out, which is correct, so it doesn't look like it's that instruction.

-[Unknown]

[dbz400]

yep , know it as vuc2i seems to be the one affect those color changes . Anyway, we've narrowed down to vfpu which mostly likely causing issues but not lighting calc issues

[unknownbrackets]

Okay, it's definitely vi2f.

-[Unknown]

[hrydgard]

Interesting.

I was thinking of one other possible cause, maybe the game would render something using the GPU and then read back the values into the CPU for some extra special lighting effect, and that might go through the VFPU for processing too so your hack would fix it, but if vi2f is buggy, then yeah, that idea can be disregarded I guess.

[unknownbrackets]

I found it, sending a pull.

There's still lots of remaining color problems and lighting issues, so there's probably more vfpu and ge bugs here.

-[Unknown]

[unknownbrackets]

There's still a weird issue even on the first screen:

It's probably the vfpu too, I guess...

-[Unknown]

[hrydgard]

No reason that can't be a GE issue, but yeah we do have more VFPU bugs, like the GTA disappearing-geometry issue.

[hrydgard]

Actually, that particular problem looks a lot like BGRA / ARGB confusion - that yellow stuff could be explained by the game using a color like ARGB #30FFFFFF (faint alpha white) but that gets interpreted as #FFFFFF30 (solid yellow).

Now, how to find where that issue arises is the "fun" part...

[unknownbrackets]

Ah, that makes a lot of sense. Well, can probably find it the same way - start at the vertex reader, if it reads in yellow, find the memory address it comes from, trace that, and then follow the instructions and memory cycles backwards...

-[Unknown]

[vontman]

Dragon ball now is working well in the last build

[dbz400]

Kingdom Heart in-game character looks okay now as well

[unknownbrackets]

Wrong colors and weird stretching and such only happen on Windows 32 bit so far (64 is fine.) Could even be another MSVC bug. Happens with jit or interpreter.

(well, not the hard color on the info dialog but after that I mean.)

-[Unknown]

[dbz400]

@unknownbrackets , do you mean 64 bit binary , the color and lighting both look correct as well as weird stretching?

[unknownbrackets]

Well, I think it's too dark (I need to compare though, I have the brightness pretty low on my monitor to preserve my eyes...) I haven't looked at it deeply but it definitely fixes the weird stretching.

The color is wrong in the initial dialog though either way.

-[Unknown]

[dbz400]

I just tried out on Android , it looks well on color , streching and dialog . Strange that only happen on win32 build

[dbz400]

Yes , it is too dark in-game , i compared with real PSP , Likely lighting issue

I think iOS should be okay as well ?

[Squall-Leonhart]

its not just the stretched banners that 64bit builds don't have a problem with

also the Disney logo on start placement and colors the "2010 Disney / Developed by Square Enix" credits overlap and colors Pre menu animation stretching and colors Title menu colors and shrunk images Blue background for text banners are cut off, also slightly too bright Status menu top banner pulled to 1/3 of the screen

Even basic colours in the game world are slightly off in 32bit builds

[unknownbrackets]

Yes, it seems to me like it's caused by some vfpu instruction being mis-emulated by MSVC (even in debug) in 32-bit only. The same happens to LittleBigPlanet videos, but using jit fixes that since we compile the instruction manually in the jit.

I haven't figured out which instruction it is in KH BBS.

-[Unknown]

[Squall-Leonhart]

I would suggest installing PVS Studio, its going to be one of those cases where the result will be different based on whether its a 32bit or 64bit build, so pvs studio should pick that up quite readily.