Project Diva Extend: Multithreading eats frames

[GamerzHell9137]

For some reason When Multithreaded is on and force real clock sync is off the game frames are being eaten. The speed is same but frames drop to 20 and lower.(In menu,in game, almost everywhere) Revision i'm using 0.9.8-1429-96916ffb

EDIT: And game in general got worse, the timing is way off. Gonna post a video later.

EDIT: As the song goes on the input is worse and worse

https://www.youtube.com/watch?v=tIU45f4MENY&feature=youtu.be

Its like the notes go faster than they should.

[RadarNyan]

As the song goes on the input is worse and worse

This is caused by Audio latency set to Low. Just use default setting (Medium) you'll be fine.

And Multithreaded option is as it says in experimental status, it just doesn't work nice with Project Diva and will make timing worse, so turn it off.

[GamerzHell9137]

@RadarNyan Have used default settings.

[unknownbrackets]

The way the game is designed, it will drop frames intentionally unless the GPU and CPU are in lockstep. There's an easy way to make multithreading do this: sync all the time.

This will make multithreading slower when on than off for every game. Woudl you like PPSSPP to change this way?

Alternatively, you can just turn it off. PPSSPP will run faster for this game, and other games can run multithreaded faster.

Specifically, the game enqueues a list without using any waits or syncs. It is specifically programmed to drop frames if the CPU goes faster than the GPU, which is exactly what the multithreaded setting does.

-[Unknown]

[GamerzHell9137]

@unknownbrackets Yeah, i guess i'll stay away from Multithreading but the issue that's in the video is still there. Tried stock 0.9.8v and it didn't had it thou the input was slow on it.

[solarmystic]

@GamerzHell9137 I can reproduce the timing issue with the game as each song progresses.

It seems like as the song goes on and on there is a significant input(?)/A/V(?) delay being introduced that gets cumulatively worse up until the very end where it just doesn't sync up with what you see anymore and trying to follow the beat will just give you "worst" and "sad" results on screen. It's very insidious since you're tempted to dismiss it at the beginning of the song when the timing is perfect, and you won't experience it unless you play each song to completion.

You basically have to hammer the notes before they even sync up with what you see on the screen at that point. This is absolutely detrimental when playing longer songs. I've never had this issue before on the older 0.9.8 dev builds using my wired 360 pad.

This issue should also be seperated out of the multithreaded one since it happens with default settings (Multithreaded OFF and force real clock sync OFF).

And you're right, the default 0.9.8 build did not have this issue. I am tempted to do a thorough bisect to see which dev build the issue started from.

Using default settings with the usual test setup:-

[solarmystic]

Okay, I think I may have found the first responsible commit for the progressively worse timing issue, which is v0.9.8-1388-gd8cff45 https://github.com/hrydgard/ppsspp/commit/d8cff4509e278d35cf2c9912fcfc2f8ad00da619 from the https://github.com/hrydgard/ppsspp/pull/6414 pull request by @unknownbrackets

Last unaffected commit from that pull request is v0.9.8-1387-g6c4a614 https://github.com/hrydgard/ppsspp/commit/6c4a614205fd45f58c24f140f94def39aa5655d5

@GamerzHell9137 Try comparing the timing on v0.9.8-1396-ga1d359f vs v0.9.8-1405-gfea0a26. Grab them both from the buildbot for the comparison, I guarantee you will notice the difference.

v0.9.8-1396-ga1d359f (before the #6414 pull request was merged to master) feels a lot more consistent, even up to the end of the songs. In fact, it's the last one I could easily get a Perfect on for the first song at Hard difficulty, which I've been doing across the older builds to test for A/V timing consistency.

@unknownbrackets You should be able to reproduce this issue using the Project Diva Extend demo itself. Any song will suffice. http://www.pspdemocenter.com/page.php?id=3907

[solarmystic]

@GamerzHell9137 Try this build which I self-compiled if you're on Windows x64:-

https://www.mediafire.com/?98nu1wwztb9hd1c

It's based on the latest master (v0.9.8-1430-gd2f4388) but with the responsible commit (https://github.com/hrydgard/ppsspp/commit/d8cff4509e278d35cf2c9912fcfc2f8ad00da619) reverted/commented out.

The timing should be close to perfect in Miku Extend for this build, I tested it myself just now:-

[unknownbrackets]

Well, I mentioned that change could definitely break things. However, I'm not sure why timing would get progressively worse unless it simply wasn't blocking audio enqueue long enough.

-[Unknown]

[GamerzHell9137]

@solarmystic Difference btw v0.9.8-1396-ga1d359f and v0.9.8-1405-gfea0a26 is day and night! v0.9.8-1396-ga1d359f works way better but your build works the best! Never was so near perfection on PPSSPP.

P.S. The only thing that i would like is to somehow make it possible to beat The Singing Passion of Hatsune Miku on Extreme, but even that song is way better on your build.

[unknownbrackets]

Can this be easily reproduced in the demo by someone who does not normally play music timing games? Maybe at the easiest difficulty or something?

Also, a JpcspTrace of: sceAudioOutput2OutputBlocking sceAudioSRCOutputBlocking sceAudioOutputBlocking sceAudioOutputPannedBlocking

Would help (using my fork of JpcspTrace here on github.) I want to see how frequently it runs on a PSP versus in PPSSPP.

What if you keep the early wake of the audio threads, and change this line (I don't think this is correct, but I want to know what affect it has):

int blockSamples = (int)chan.sampleQueue.size() / 2 / chanQueueMinSizeFactor;

To:

int blockSamples = ((int)chan.sampleQueue.size() / 2 / chanQueueMinSizeFactor) * 10;

Does it change the timing of the audio?

-[Unknown]

[solarmystic]

@unknownbrackets Yes, you can easily reproduce it. Just choose the easiest difficulty and start any of the demo songs. Play it for a bit, and over the course of the song, you'll realise that the timing is off with the button presses as compared to what you're hearing/seeing on the screen.

Unfortunately it's not something that you'll experience until you're at least partway through a song, so you'll have to play through it. It's somewhat of a cumulative effect, and is also rather subtle, strangely enough

[solarmystic]

@unknownbrackets Changing

int blockSamples = (int)chan.sampleQueue.size() / 2 / chanQueueMinSizeFactor;

to

int blockSamples = ((int)chan.sampleQueue.size() / 2 / chanQueueMinSizeFactor) * 10;

at line 200 of the __sceAudio.cpp file completely wrecks the audio output of the game (any game in fact), heh. Everything is being played back in super slow motion and it's ear wrecking static.

EDIT:-

You can increase the number at the end of the

int blockSamples = ((int)chan.sampleQueue.size() / 2 / chanQueueMinSizeFactor) *

line to around 2 before the audio is just incoherent. 3 and above are just painful affairs.

[unknownbrackets]

You can change the 10 (maybe 4 or 5), but I'm most interested to know if that changes how it gets progressively worse. Sorry for not mentioning, I knew it would wreck the audio quality.

-[Unknown]

[solarmystic]

@unknownbrackets Oh, in that case, changing it to 4 or 5 does not seem to change how it gets progressively worse. But the audio is so far behind the visuals (like I mentioned earlier, super slo mo audio) with that modification that I can't even play the game properly enough to even completely ascertain that. I think we're barking up the wrong tree.

[unknownbrackets]

Well, I just played through an easy song. I lost horribly (58% I think it says?), but I'm pretty sure I got to the end. This isn't because of poor timing but because I don't play these sort of games. The last part was "chance time" with rainbow things on the circles, if that confirms whether I got far enough or not.

Anyway, she raised her knee to the music and there was a single circle to hit at three points in the song (iirc? or was it four?), including right at the end of "chance time". Seemed like the stanzas of the song I guess. Each of the three times it seemed to align fine with the music. At least not far enough off for me to notice it.

This was with latest master.

-[Unknown]

[solarmystic]

@unknownbrackets Like I suggested @GamerzHell9137 do earlier on, try comparing the timing on v0.9.8-1396-ga1d359f (without the early audio wakeup commit) vs v0.9.8-1405-gfea0a26 (with the audio wakeup commit). You will notice a visible difference.

I guess on easier modes it's harder to notice it, since the timings are a lot more flexible (less buttons on the screen to push, etc.).

[GamerzHell9137]

@unknownbrackets There's no way to not see the difference btw v0.9.8-1396-ga1d359f and v0.9.8-1405-gfea0a26.

Or just check solarmystic's build which makes the game totally fine https://www.mediafire.com/?98nu1wwztb9hd1c

[unknownbrackets]

Second time, with it commented out. I did better (79% "CHEAP"), but the timing did not seem severely different to me. I just did better because I remembered the song better.

I tried it again, without it commented out, and a controller. I got 69%, had trouble with the parts which were several taps in a row. Still seemed like the same timing.

You're saying it's completely out of whack. I'm not saying you're wrong, but this level of completely out of whackness is simply not enough for me to notice. How far off? If it's off between these two, it's gotta be by a matter of frames. What you were saying sounded originally like a whole second+ off which I would be able to notice.

I've already played it 3 times, and I don't like this kind of game. I'm not gonna play it again without good reason.

-[Unknown]

[GamerzHell9137]

@unknownbrackets Played first on solarmystic's build then on the latest PPSSPP version as u can see

99% on solar's build and

[unknownbrackets]

Well, if it's off by a single frame or something, I can't help you. I've never had sharp ears, and I have never liked these sorts of games. Unless this is affected by CPU or GPU or something. It'll take someone debugging this who can tell.

It's clear from testing on a PSP and other games that better-priority threads do indeed schedule. So something else is wrong in this game. It doesn't seem possible for me to tell if it's fixed or not, so I won't be able to tell what.

It sounds like you're saying that the audio plays earlier than the graphics at the end of songs?

-[Unknown]

[GamerzHell9137]

@unknownbrackets The song and animations play fine but it seems like the notes that ur supposed to press go faster than they are supposed too or something like that.

[unknownbrackets]

Does commenting out currentMIPS->downcount -= 1200; in sceKernelThread.cpp affect it? That's probably the only difference in timing here; where it's eating the cycles to switch.

You can try this if that helps, maybe it makes sense:

    // Switching threads eats some cycles.  This is a low approximation.
    if (fromIdle) {
        // Don't eat any cycles when leaving idle, assume it was eaten during idle.
    } else if (toIdle) {
        currentMIPS->downcount -= 1200;
    } else {
        currentMIPS->downcount -= 2700;
    }

-[Unknown]

[GamerzHell9137]

@unknownbrackets Ur gonna need to wait for solarymystic cause idk any of the stuff you are talking guys. Off topic question, where do u learn this kind of stuff? Now more on topic, i'll try to slowdown the footage so you can see what's wrong.

[unknownbrackets]

When I learned programming, there were quite a few other people on the Internet talking about QuickBASIC. A search on Dogpile would lead you to a webring that most of the interesting websites (and many uninteresting ones) were connected to. Then I just read stuff and tried it. My first program was a horrible concoction of a journal system that was DOS-only and didn't even have save capabilities. But it could remember things you typed when you went back to the menu. It was quite buggy, and I worked on it together with someone else even.

QuickBASIC also had an online help that was pretty good. Nothing as good as the Internet, mind you, but good all the same. Honestly, the only thing harder about programming these days is that there's so much info and examples available out there it's probably harder to know where to start. But there's plenty of tutorials, no need to be picky.

-[Unknown]

[solarmystic]

@unknownbrackets Commenting out currentMIPS->downcount -= 1200; in sceKernelThread.cpp and keeping the https://github.com/hrydgard/ppsspp/commit/d8cff4509e278d35cf2c9912fcfc2f8ad00da619 (audio drain commit) intact seems to have helped to rectify the situation.

Subsequently, using the suggested code in your post (https://github.com/hrydgard/ppsspp/issues/6441#issuecomment-47496636) also helps to resolve the situation. Completely, and with the audio drain line (https://github.com/hrydgard/ppsspp/commit/d8cff4509e278d35cf2c9912fcfc2f8ad00da619) intact and uncommented.

@GamerzHell9137 Here's another newer build with those changes for you to try out, you can help me verify whether or not the changes @unknownbrackets suggested helped out:-

https://www.mediafire.com/?qd35k5luq1xsa0f

[GamerzHell9137]

@unknownbrackets Find programming in general quite awesome. I'm going to college and will learn a bit of it there (iirc the professor said C++ is the language that will be studied there and stuff that Linux is the future :P ) and i hope i will continue it cause i find it really interesting.

@solarmystic And have just tried the build

and its fine, there's nothing wrong with it but now idk if the 1st build is better or the second one. They are almost the same so i can't really say which one is better (I guess the latter)

And a bit late but here was the problem with the new PPSSPP builds and the difference btw Solar's 1st build.

https://www.youtube.com/watch?v=r-OUyEctOXw&feature=youtu.be

[solarmystic]

@GamerzHell9137 That's good. The two builds I uploaded are supposed to exhibit the same behaviour in the game, which means that the issue is fixed with the changes [Unknown] proposed. Thanks for the feedback.

[GamerzHell9137]

@solarmystic One more thing, when there are 3 fast or more notes that come one after another,the controls get a bit wacky, not sure if you see it. Its in songs Singing Passion, Look this way Baby, Palette and so on. It might be only me but i still can't beat Singing Passion on Extreme cause the controls ( me=issue?) can't handle the speed.

EDIT: I remember on really old builds i was possible to beat it somehow.

[unknownbrackets]

What if you change this function in CoreTiming.cpp instead?

void ProcessFifoWaitEvents()
{
    while (first)
    {
        if (first->time <= GetTicks())
        {
//          LOG(TIMER, "[Scheduler] %s       (%lld, %lld) ", 
//              first->name ? first->name : "?", (u64)GetTicks(), (u64)first->time);
            Event* evt = first;
            first = first->next;
            event_types[evt->type].callback(evt->userdata, (int)(GetTicks() - evt->time));
            FreeEvent(evt);
        }
        else
        {
            break;
        }
    }
}

(the change is GetTicks().)

-[Unknown]

[solarmystic]

@unknownbrackets I get a

CoreTiming.cpp(516): warning C4018: '<=' : signed/unsigned mismatch

whilst compiling ppsspp with that change, is that normal?

Anyway, changing that particular function (while keeping the others default and https://github.com/hrydgard/ppsspp/commit/d8cff4509e278d35cf2c9912fcfc2f8ad00da619 active) does not help to resolve the issue. It's back to square one again, the difference is noticeable as compared to the builds produced with the prior suggestions in https://github.com/hrydgard/ppsspp/issues/6441#issuecomment-47496636

[hrydgard]

@unknownbrackets , internet and webrings? Luxury! In my day, we had to walk uphill both ways to the library to borrow books about BASIC programming for the wrong computers, and had to try to adjust the code to fit GWBASIC and QBASIC, not always with success...

That signed/unsigned mismatch is probably not much to worry about.

[unknownbrackets]

Heh. Well, I'm still not 100% sure the idle thing is right, hmm...

-[Unknown]

[GamerzHell9137]

@solarmystic It seems with your 2nd build Project Diva 2nd is fixed too

[unknownbrackets]

Possibly 5cd4a17e885e3ae1280f74f6e77fd620a58c57a7 is the correct fix. Even if not, it was definitely a problem.

-[Unknown]

[solarmystic]

@unknownbrackets I'm not quite sure whether https://github.com/hrydgard/ppsspp/commit/5cd4a17e885e3ae1280f74f6e77fd620a58c57a7 conclusively did it, but I think it helped out somewhat, I don't feel the timing "drift" anymore using the latest master from the buildbot (without the prior modifications suggested in this thread):-

[GamerzHell9137]

@solarmystic Thou Project Diva 2nd likes ur second build more

Have played the game simultaneously on Master Build and Solarmystic's 2nd Build

Left - Solarymystic's 2nd Build Right - Master Build

[solarmystic]

@GamerzHell9137 I agree with your assessment, that build I posted with the following suggested change by [Unknown] in sceKernelThread.cpp

    // Switching threads eats some cycles.  This is a low approximation.
    if (fromIdle) {
        // Don't eat any cycles when leaving idle, assume it was eaten during idle.
    } else if (toIdle) {
        currentMIPS->downcount -= 1200;
    } else {
        currentMIPS->downcount -= 2700;
    }

is still better as compared to the current (v0.9.8-1515-ge505655) master in Diva 2nd. I thought it was the placebo effect but your results have shown otherwise.

But the way the files have changed now since the most recent commits would mean that the aforementioned change in sceKernelThread.cpp may not apply correctly to future builds anymore in these games.

[unknownbrackets]

The more I think about the idle thing, the less I think it makes sense. Anyway, it was probably just drifting differently. I don't like this whole "if I win, it's correct" method of testing.

Anyway, the us interval that's been involved in those changes is: audioIntervalUs = (int)(1000000ULL * hwBlockSize / hwSampleRate);

It happens that this is 1451.2472us. Notice that it's not an integer. So if this game is that sensitive to the timing interval, it's probably still an issue. Right now it may play the audio 0.0002472 more seconds earlier than it should every second. That means in just over 4045 seconds (> 1 hour), if the game uses the audio interval solely for timing, it might be 1 second off (early.)

The change altered the drift but didn't remove it, which probably sometimes offset the above negative drift in a "lucky" way.

Theoretically, it would be relatively easy to make it only 0.00000392207 seconds off (1/222 as much), but theoretically the drift could be accounted for by just tracking how much of it there is.

-[Unknown]

[solarmystic]

@unknownbrackets I agree that the current method of verifying whether or not the solution worked via the results of repeated playthroughs of a given song in the game is "unscientific" and very subjective to say the least, but there's currently no other practical way that I can think of, as an end user.

You've only got your gut to rely on when it comes to these games, and when you've spent countless hours using prior builds to play the game when it just worked, you get a "feel" for it when it just doesn't anymore after a certain build, trite as it may sound. Muscle memory is a real thing.

That's how I did the bisect for it in the first place. There's no other way to cut through the builds otherwise, and the fact is that the results can be replicated reasonably well by another user means that the issue is real and not imagined.

[unknownbrackets]

Anyway, those things aside - for multithreading, even at 8x+, I get pretty much solid 30 fps in game. Only in the music selection dialog does it drop frames sometimes, which it may but probably doesn't do on a PSP, and it doesn't seem to be very noticeable there.

Does it affect gameplay with recent changes and multithreading on?

-[Unknown]

[solarmystic]

It's still dropping to as low as 11 FPS in the music selection dialog screen, but during gameplay itself the drops are less severe. (25 is the lowest I've seen it go).

Playing on harder difficulty levels, even the slightest framedrop is unacceptable to the experience.

This is with multithreading enabled of course. Speed is always 100% in all cases, it's the internal FPS itself that is dropping.

[unknownbrackets]

The game itself really is dropping frames, and the way it's written, it most likely would drop frames on a PSP as well (if it were e.g. drawing at 960x544... well, most games are written such that they would drop frames in that situation, but this game is written in a different way.) This will probably be affected by your GPU and CPU performance, and your graphics settings.

It really goes back to a choice: why are you using multithreading? I don't see how it's possible to make this game show 30 fps with multithreading enabled unless:

• You have a very fast GPU and we don't need to change any code.
• ALL games, including this one, run more slowly with multithreading on than with it off (e.g. < 100% speed.)
• You're using Linux or perhaps Mac OS X, which is capable of timing things with sub-millisecond accuracy.

I doubt most users will be happy if multithreading is slower, we should probably simply remove the option in that case. So we can retitle this issue to "Please remove the multithreading option in PPSSPP."

-[Unknown]