Ppsspp is non-determenistic

[InfamousKnight]

Rng is system time based meaning like January 12 6:00am is what the game sees. This applies to real psp, which is why they’re not fixing it. You can get around it by loading a save state, but according to keylie, it should make it deterministic either way.

Desyncs are also pretty random. Like even with the state loaded, inputs seem to do what they want at times. Different things happen at certain times(mostly later).

To reproduce, follow the known sync settings along with editing ppsspp.ini on line frameskipunthrottle to false and run like Star Wars battlefront 2 and get to a map and move around and each time you play it back, you will end up somewhere else in the same area. I’m pretty sure ff3 will give you the similar results, beat the first battle, replay it a few times, and different things will occur.

[InfamousKnight]

Sorry if I sound pushy here, but has anyone been able to at least reproduce the above? Better yet a solution?

[clementgallet]

Yes, I confirm that the rng in Star Wars Battlefront 2 is not deterministic. I'm looking at it.

[InfamousKnight]

Further testing with pspkvm, I wanted to tas driv3r Java and managed to complete the Miami segment with no rng related desyncs. It’s still prone to desyncs, sometimes you have to replay it twice. And I haven’t tested it without the save state load on the emulator and still have main thread clockgettime checked. Haven’t tested without those. Encoding it seems to be a leap of faith.. Miami was completed in ~30 seconds. Stuck on first mission in France right now.. resyncing it at a spot where it always desyncs seem to work fine.

I have to work for the rest of the day in like 30 minutes.. I’ll do more testing tomorrow.

[clementgallet]

Honestly, I don't know what can cause an emulator to be non-deterministic (when ruling out the system time thing), so I don't know what to look for.

[InfamousKnight]

Oh well. It might be game dependent. Like maybe some psp games use their own algorithm or something. Emulators on psp should. So maybe pspkvm would be deterministic.

[InfamousKnight]

More testing done last night with battlefront 2, I wanted to see if I could manage without the emu save state. I had no idea what I was doing, but I enabled everything in main thread time tracking and seemed to get some good luck. For some reason, starting location was becoming random unlike before. Like where you start on the map. Not in the same area like the bar. And all I had to do was hope I end up in the bar each time I play back. I played it back like 10 times and got lucky 7 out of 10 times. The same character spawns too when I end up in the bar. Rarely will he spawn in a different spot..

More testing this morning, and to get around the map not loading in the right spot, you just nudge it on the boundary. So if I wanted to spawn in the bar, just keeping pressing right like 3 times. Then you will guarantee you will end up there... but randomness is still there.. both enemies and allies don’t spawn in the same location each time and inputs just go out of sync that have nothing to do with rng..

This is so disappointing..

[InfamousKnight]

Another problem with this, is gran turismo and need for speed underground rivals just crash after splash screen. They work natively(I also double checked if opengl was checked). So I thought a racing game must not involve much rng right? I couldn't reach the profile screen. Also with rivals if it crashes 3 times, and you play it back, it will freeze like 30 before the last frame. This is as complicated as amiga is..

[InfamousKnight]

So I'm afraid you haven't gotten anywhere with this?

[qixils]

I'd imagine emulators with issues not seen in any other games aren't very high on the priority list, especially with no ideas on where to start looking for fixes...

[InfamousKnight]

Why not save system time in save states? Then fake the system time and return to real system time while not running anything if that makes any sense.

[qixils]

libTAS's emulated system time is saved in save states.

[clementgallet]

Please be patient about this. I'm (obviously) working on this on my free time, and I have currently none. Also, this issue is probably more a PPSSPP issue than a libTAS issue, and fixing it requires to dig into PPSSPP source code, which takes a lot of time.

Also, addressing this issue to PPSSPP devs, without concrete elements to look for, will probably be not helpful. I guess this kind of issue has a very low priority to them, so they won't spend much time on it.

[clementgallet]

It looks like I managed to get Star Wars Battlefront 2 to behave deterministically. I modified the OpenGL backend so that it does not run in a separate thread, and I made the PSP ISO loading to run in the main thread as well. After these changes, PPSSPP is single-threaded, except for the GameInfoCache object that is using a WorkQueue, but it doesn't seem to affect sync (I could modify it if necessary).

[InfamousKnight]

Is there a modified build then?

On Tue, Apr 2, 2019, 12:06 PM Clément Gallet notifications@github.com wrote:

It looks like I managed to get Star Wars Battlefront 2 to behave deterministically. I modified the OpenGL backend so that it does not run in a separate thread, and I made the PSP ISO loading to run in the main thread as well. After these changes, PPSSPP is single-thread, except for the GameInfoCache object that is using a WorkQueue, but it doesn't seem to affect sync (I could modify it if necessary).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/clementgallet/libTAS/issues/151#issuecomment-479070262, or mute the thread https://github.com/notifications/unsubscribe-auth/ACfLcQCogot_d5m7Uw2vDpnS10AFxqWAks5vc3_-gaJpZM4aAKqD .

[clementgallet]

Yes, I'm writing a PR, we'll see.

[InfamousKnight]

Cool. I'm glad we finally found a solution. Sadly I work so much..

[InfamousKnight]

Wouldn't it be faster and easier just to upload some libtas only build of ppsspp? Ppsspp has good enough capability with just about everything at this point. In fact, someone should also update the game capability page under ppsspp with a link to the libtas only build.

Again, sorry if I sound pushy.

[clementgallet]

Well, there's TASVideos rule:

Every officially supported TAS emulator has a repository to host official releases obtainable as a package (like binaries). These official releases are supported by TASVideos. Using custom or interim builds compiled from emulator source code (from e.g. svn or git repositories) is not officially supported. Always make sure your movie syncs on the official releases; use interim builds at your own risk. If a movie syncs on some interim build, but doesn't sync on any official release, it will be rejected.

This is a bit different, because the "emulator" (libTAS) is an official release, but not the "game" (PPSSPP). Of course, we don't have to follow TASVideos rule to make TASes, and exceptions could be made. I will push the modified PPSSPP on my fork, so you can use it.

[clementgallet]

https://github.com/clementgallet/ppsspp contains the patch to make most of the execution single-threaded. You need to add the option EnforceSingleThreaded = True in ppsspp.ini, in the General category.

[InfamousKnight]

Thanks! Now how do you compile it?

On Wed, Apr 3, 2019, 1:57 PM Clément Gallet notifications@github.com wrote:

https://github.com/clementgallet/ppsspp contains the patch to make most of the execution single-threaded. You need to add the option EnforceSingleThreaded = True in ppsspp.ini, in the General category.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/clementgallet/libTAS/issues/151#issuecomment-479594867, or mute the thread https://github.com/notifications/unsubscribe-auth/ACfLcW4vhraIP8X0cf2ML-ygckCmKcqtks5vdOuAgaJpZM4aAKqD .

[clementgallet]

https://github.com/hrydgard/ppsspp/wiki/Build-instructions

[InfamousKnight]

I'm getting a build error after 84% replay.cpp /home/seab/ppsspp/Core/PSPLoaders.cpp: In function ‘bool Load_PSP_ISO(FileLoader, std::__cxx11::string)’: /home/seab/ppsspp/Core/PSPLoaders.cpp:259:7: error: ‘function’ is not a member of ‘std’ std::function<void()> execLoaderFunc = [bootpath] { ^~~~ /home/seab/ppsspp/Core/PSPLoaders.cpp:259:7: note: suggested alternative: ‘is_function’ std::function<void()> execLoaderFunc = [bootpath] { ^~~~ is_function /home/seab/ppsspp/Core/PSPLoaders.cpp:259:24: error: ‘execLoaderFunc’ was not declared in this scope std::function<void()> execLoaderFunc = [bootpath] { ^~~~~~ At global scope: cc1plus: warning: unrecognized command line option ‘-Wno-deprecated-register’ CMakeFiles/Core.dir/build.make:4886: recipe for target 'CMakeFiles/Core.dir/Core/PSPLoaders.cpp.o' failed make[2]: [CMakeFiles/Core.dir/Core/PSPLoaders.cpp.o] Error 1 make[2]: Waiting for unfinished jobs.... CMakeFiles/Makefile2:564: recipe for target 'CMakeFiles/Core.dir/all' failed make[1]: *** [CMakeFiles/Core.dir/all] Error 2 Makefile:129: recipe for target 'all' failed

[clementgallet]

Sorry, it was a missing header, but somehow it still compiled fine for me. I pushed the fix.

[InfamousKnight]

okay.. now there is a second problem.. The game screen turns black when I enforcesinglethreaded. If I turn it off, it works. You can hear the sound in the background, it justs nothing renders

More info, this happens when running natively.

[InfamousKnight]

I just realized I was running a shared library. And it only opened through the command line. How do you patch an executable?

[clementgallet]

I don't understand what you are saying about the shared library. The black screen issue might be because of other options that I changed, I will see what to change from a clean option file.

[InfamousKnight]

Well, when I compile it, it shows a shared library instead of a executable. Which can only be ran through the command line or libtas. Somehow I thought it would make a difference.

[clementgallet]

Don't you have a PPSSPP executable in the build directory? How are you determining that a certain file is a shared library?

[InfamousKnight]

I have PPSSPPSDL in the build directory. I determine it's a shared library by right clicking and showing properties.

I got the ppsspp folder by git clone --recurse-submodules https://github.com/clementgallet/ppsspp.git

If you just download the zip, install depencies, run ./b.sh, there will be build errors regarding ffmpeg. Looking up this problem led to executing the aforementioned command with git

[clementgallet]

Ok, I think it's fine.

[InfamousKnight]

Um, has the graphical issue been fixed?

[qixils]

i don't know what issue you are referring to but it is almost certainly not a fault of libtas and is therefore probably not relevant here. i presume whatever graphical issue you have is either PPSSPP having an issue in emulation or llvmpipe not supporting something in its software rendering. if you can determine it is the former, i'm sure the PPSSPP devs would love to hear about it. if it is the latter, i don't believe anything can currently be done.

[clementgallet]

Again, please be patient, you only have raised this issue two days ago.

Setting RenderingMode option to 0 should fix the issue. I also suggest to set SeparateSASThread and SeparateIOThread to False.

[InfamousKnight]

Yeah, sorry for being pushy about this.

To make up for it, I figured out what to do:

Set rendering mode to 0 and libtas time tracking gettimeofday enable and it should sync each time with the same result.

Thank you so much for fixing this! I also tried leaving the thread options to true and it still works.

On Sat, Apr 6, 2019, 2:34 PM Clément Gallet notifications@github.com wrote:

Again, please be patient, you only have raised this issue two days ago.

Setting RenderingMode option to 0 should fix the issue. I also suggest to set SeparateSASThread and SeparateIOThread to False.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/clementgallet/libTAS/issues/151#issuecomment-480527196, or mute the thread https://github.com/notifications/unsubscribe-auth/ACfLcaESZNLPuu-Mdy0XvX1GgHgzbCUlks5veOi1gaJpZM4aAKqD .

[InfamousKnight]

PSPKVM has some desync problems. It works fine if you don't use frame advance or slow motion, but then its not really a tas. Which is odd how that's the only problem this time.

[clementgallet]

Looking into other emulator's code is tedious, and also those keep updating so I can't keep up anyway.