Collect probably IO timing related problems

daniel229 commented 9 years ago

Meaning problems got help by changing I/O timing method to "HOST" or"Simulate UMD delays".

"Simulate UMD delays" helps the following games

Innocent Life https://github.com/hrydgard/ppsspp/issues/2830
Dan Ball Senki W https://github.com/hrydgard/ppsspp/issues/4786
Harukanaru Toki no Naka de 3 with Izayoiki Aizouban https://github.com/hrydgard/ppsspp/issues/6127
DJMAX Portable 2 https://github.com/hrydgard/ppsspp/issues/5837 ,probably also helps other DJMAX games.
Super Robot Taisen OG Saga: Masou Kishin II https://github.com/hrydgard/ppsspp/issues/5540
Mahou Shoujo Madoka Magica Portable https://github.com/hrydgard/ppsspp/issues/6557
Gran Turismo https://github.com/hrydgard/ppsspp/issues/6189
Crash: Mind Over Mutant https://github.com/hrydgard/ppsspp/issues/7645
Tengai Makyou - Daishi no Mokushiroku (hangs on certain scene)
Pangya: Fantasy Golf https://github.com/hrydgard/ppsspp/issues/4996
Project Cerberus #2182
Castlevania X Chronicles(Rondo of Blood) #11091
Driver 76 #12054
Bust-A-Move Deluxe(videos) #8795
Arctic Adventures: Polar's Puzzles #12677
Aces of Wars #11062

"Host" helps the following games

Ace Combat X2 Joint Assault https://github.com/hrydgard/ppsspp/issues/5545
Final Fantasy 3 #10588

"Simulate UMD delays" breaks the following games

Radiant mythology2 (hang)
Ghost in the shell (hang)
Race Driver 2006 (hang) https://github.com/hrydgard/ppsspp/issues/7425
world neverland (slow)

unknownbrackets commented 9 years ago

The current logic is really basic, it's just this:

        if (abs((int)lastReadBlock_ - (int)secNum) > 100) {
            // This is an estimate, sometimes it takes 1+ seconds, but it definitely takes time.
            usec = 100000;
        }

More correctly, the time on a UMD will vary based on:

Distance from center of UMD.
The actual change in physical position (e.g. track -> track and sector -> sector.)
Rotation speed of UMD (e.g. spin up delay.)
Whether the data is in cache (PSP-2000+.)

For example, World Neverland might be reading things very efficiently with things laid out on separate tracks that are near each other. Or our delays are just too high for it.... or, it might simply run slow on PSP-1000 devices, I guess (probably not.)

However, there's a simpler solution: if all of these games work on a PSP as iso/cso files on a memory stick, then presumably it can't depend on the above factors of rotation speed and whatever. It's probably a lot easier to simulate the access speed and latencies of flash storage (we already try, but we're a bit too aggressively fast.) I don't think that layer has "seek delay" emulation, so theoretically we shouldn't need it either.

-[Unknown]

daniel229 commented 9 years ago

Pangya: Fantasy Golf won't work on memory stick with cfw 660pro-c2

vnctdj commented 9 years ago

The demo of Monster Hunter Freedom Unite has a black screen after language selection. Using "Simulate UMD delays" or "Host" fixes the issue.

ppmeis commented 9 years ago

Confirmed working DJ MAX titles with Simulate UMD Delays option.

daniel229 commented 9 years ago

~~"Simulate UMD delays" helps the random hang in Youkoso Hitsuji-Mura Portable~~ seems like a translate patch bug,and not really help.

hrydgard commented 9 years ago

It already seems it helps more games than it breaks, so I might change the default soon.

LunaMoo commented 7 years ago

Don't have the other marked as hanging with "Simulate UMD delays", but GitS:SAC works fine with it for me.

vnctdj commented 7 years ago

The default setting for "I/O timing method" is still "Fast (lag on slow storage)". Therefore I think this issue should be updated. Does "Simulate UMD delays" still break some games ?

CookiePLMonster commented 5 years ago

I'll run more tests too, but it would seem like Driver 76 should be added to the list of games for which "Simulate UMD delays" helps, as per #12054.

ghost commented 5 years ago

I use "Simulate UMD delays" for the 4 games I own but only Castlevania Dracua X benefits from it I guess. I don't think Lunar Silver Star Harmony, Mega Man Powered Up or Maverick Hunter X is dependent on it but they work OK still.

Would changing the default value "Fast" to "Simulate UMD delays" be that bad today? It's been 4 years since this issue surfaced after all.

unknownbrackets commented 5 years ago

I took a stab at async IO using threads. It kinda works, but for example LittleBigPlanet gives errors about calling pthreads on non-pthreads (within the game, not host pthreads.) It doesn't seem related to callbacks...

https://github.com/hrydgard/ppsspp/compare/master...unknownbrackets:io-async

-[Unknown]

sum2012 commented 5 years ago

@unknownbrackets I still get invalid address in Mahou Shoujo Madoka Magica Portable in beginning story in your change.

debug log: https://drive.google.com/file/d/1UFuDEcIW0SzEdOr35iXbAYIdEZ1s229o/view?usp=sharing

unknownbrackets commented 5 years ago

I made sure the thread waits for the entire IO operation, and fixed some file open issues. This seems to have made the branch more stable - at least, I'm no longer seeing errors in the chunk of games I tested.

https://github.com/hrydgard/ppsspp/compare/master...unknownbrackets:io-async

-[Unknown]

hrydgard commented 5 years ago

Neat, sounds promising.

vnctdj commented 4 years ago

For reference: link to the merged pull request.

unknownbrackets commented 4 years ago

It'd be interesting to validate if "Simulate UMD Delays" is no longer needed for those issues, or no longer breaks those games.

Host is probably always going to be a wildcard, but if those games no longer are helped, that'd be useful information too.

-[Unknown]

vnctdj commented 4 years ago

The demo of Monster Hunter Freedom Unite has a black screen after language selection. Using "Simulate UMD delays" or "Host" fixes the issue.

Now "Host" behaves the same as "Fast": a blocking black screen. It means that only "Simulate UMD delays" fixes the issue.

unknownbrackets commented 4 years ago

The behavior of Host is non-deterministic. Meaning:

With "Fast", you'll get the same bugs every time you try, on every device.
With "Simulate UMD Delays", you'll get the same bugs every time you try, on every device.
With "Host" everything depends on the device, whether OS has cached things, and activity from other apps on the device. You may get a bug one day and not be able to reproduce it the next. It may fix things on one device, but break them on others.

-[Unknown]

vnctdj commented 4 years ago

Thanks for your explanations, but I'm having trouble understanding the interest of a non-deterministic option? Isn't it confusing for the user?

hrydgard commented 4 years ago

Host mode will provide the fastest possible non-blocking loading, I think. Maybe it's not worth it though, I would be open to getting rid of it.

unknownbrackets commented 4 years ago

Well, Host is never faster than Fast. It mostly exists for slow storage.

Example: let's say I'm streaming the disc via network from my PC. Usually, my connection is fast but occasionally it may get slow due to wifi signal noise nearby, etc.

In that situation, Fast and Simulate UMD Delays would just stall and hang if the storage was slow. Music would stop playing, loading screens would stop spinning, and emulation would grind to a halt. That's the cost of determinism.

In contrast, Host will take as long as it takes. Again, never less time than Fast. In some ways, this is realistic. On a PSP, if the UMD had stopped spinning, a disc read would potentially take a couple seconds to fulfill. So games are often prepared for an IO operation to be slow: music keeps playing, you see loading animations, and everything works more or less as it should.

It can be faster than Simulate UMD Delays, but that will not be the common case. Most (or all) operations will be slower than Simulate UMD Delays.

So Host is not really bad imho, it's just not deterministic.

If we removed it, it'd significantly impact the usability of remote disc streaming.

-[Unknown]

hrydgard commented 4 years ago

Argh, yes, thanks for setting me straight. I was mainly thinking of the non-blocking nature of it.

Yeah, not getting rid of it.

Panderner commented 4 years ago

It seems some games like F1 2006 and Arctic Adventures: Polar's Puzzles forced to switch simulated UMD delays via compat.ini

sum2012 commented 4 years ago

It seems Arctic Adventures: Polar's Puzzles need Simulate UMD Delays in https://github.com/hrydgard/ppsspp/issues/12529#issuecomment-593532576

hrydgard commented 4 years ago

It shouldn't need it though as it's a mini and doesn't run from UMD... Very strange, probably accidentally covers some other problem..

dankcushions commented 4 years ago

perhaps for devices with low storage access speeds and RAM to spare, a mode where it pre-caches the entire ISO into RAM at start and then it could presumably safely use 'fast' or 'simulate UMD delays' mode. i think beetle-psx does something like this: https://github.com/libretro/beetle-psx-libretro/commit/ee40afa0ca01f5d501277c64ec9cafcf19539c70

PSP ISOs are about 500MB, right? quite a lot of devices could probably spare that amount of RAM.

unknownbrackets commented 4 years ago

We already have an option to preload the entire ISO into RAM. PSP games are often as much as 1.8 GB. I don't believe this option helps any of the issues linked here, though.

This issue doesn't really have anything to do with device speed. It has to do with games that accidentally contain bugs, and malfunction when the IO timing doesn't match what happened on a PSP.

For example, on a real PSP the disc spins down after not being used for a while (in part because it's silly loud, but it also burns battery to spin it.) Maybe a game developer "knew" that the disc would always need 2 seconds to spin up when certain code ran, so just assumed that the IO could not possibly complete for 2 seconds.

A PS1 has simpler storage emulation needs. PSP games have to deal with:

Some games were only ever played from Memory Stick storage, and would have bugs or be slow if treated like they ran off a disc.
Some games could run off a disc OR storage, and might have the same game ID but subtly different code.
PSP-2000 devices and higher had a cache that made repeated reads and possibly nearby reads much faster. This was in part a reaction to the terrible and frequent stalls games could have.
As mentioned, the UMD could spin down during game play. Spinning back up took a while.
Seeking on the UMD was sometimes slow.

Most of the above problems are problems a PS1 emulator doesn't have. It just has to emulate a CD-ROM drive (which can still have complex timing, don't get me wrong.)

Currently, PPSSPP doesn't do much of the above, though. It:

Always emulates games as if they ran from the Memory Stick. Note: this is essentially the same as CFW would act.
Roughly models timing using a microSD RAID-0 adapter with two UHS-1 cards. Some users used high speed cards like this, but systems came with MUCH slower class-4 speed Memory Stick cards.
Has a mode (Simulate UMD Delays) that takes the above, and tries to add a really crude seek delay emulation.
Has another mode that takes the above (not the seek delay), and waits for host storage - this is really only designed for networking. But sometimes it "fixes" bugs, because it allows reads to take longer. This probably just indicates a bug in the IO modeling (i.e. if the device can't read the content that fast and the game glitches if it's read that fast, it's probably simply too fast.)

We currently do nothing to emulate:

Disc spin up times - which were very annoying, btw, when actually playing games. I can absolutely guarantee users would complain and file bugs if we accurately emulated them.
Any part of the UMD cache, which is faster than the "Fast" timings we have if it has the data.
Correct UMD spin / seek behavior.
Any differentiation on games from UMD vs Memory Stick.
Slower Memory Stick card speeds (possibly as an option) so users can enjoy loading screens for longer.
The very loud noises (sometimes enough to drown out the music/sound effects) and device vibration that occurred when the UMD spun up or seeked to new content.

We also do some other things, such as CPU operations, HLE syscalls, and GPU drawing faster or slower than we should. This can interact with IO timing.

-[Unknown]

sum2012 commented 2 years ago

Mahou Shoujo Madoka Magica Portable is solved by #15930 Edit,it is wrong change

sum2012 commented 10 months ago

Welcome to test new IO timing method "Simulate UMD slow reading speed" in any game

hrydgard / ppsspp

Collect probably IO timing related problems #7647