search

hrydgard / ppsspp

A PSP emulator for Android, Windows, Mac and Linux, written in C++. Want to contribute? Join us on Discord at https://discord.gg/5NJB6dD or just send pull requests / issues. For discussion use the forums at forums.ppsspp.org.
https://www.ppsspp.org
Other
11.37k stars 2.18k forks source link

Misc Ad-hoc issues part 2 - input lag issues #15336

Open ghost opened 2 years ago

ghost commented 2 years ago

Game or games this happens in

List of games that may experience abnormal input delay:

Hokuto no Ken: Raoh Gaiden - Ten no Haou - Only reproduced when playing over Linux/Android (POSIX socket related OSes) , the game works fine as intended on Windows. The inputs are delayed for the 2nd instance/player which eventually would cause a desync/disconnection to happen for both players involved. Videos for the issue and the log in real time:

https://user-images.githubusercontent.com/33392696/151391098-448b3def-b7ed-4459-94c5-0bb4024b2ddb.mov

https://user-images.githubusercontent.com/33392696/151391135-a99465e4-caeb-47ab-b28d-fbcce077743b.mov

Kidou Senshi Gundam Seed: Rengou vs. Z.A.F.T. - Input lag increases over time

For both games , Force real clock sync was used here and the game runs fullspeed for everyone.

What should happen

Hokuto no Ken: Raoh Gaiden - Ten no Haou - game shouldn't disconnect and have delayed inputs.

Logs

Posted a log for hokuto no ken in this issue before: https://github.com/hrydgard/ppsspp/issues/14554#issuecomment-933623013

Platform

Linux / BSD

Mobile phone model or graphics card

Radeon 530

PPSSPP version affected

v1.12.3-733-gc9dee44cf

Last working version

No response

Graphics backend (3D API)

Vulkan

Checklist

ghost commented 2 years ago

Pursuit Force seems to have graphical issues but I don't think it factors into the input lag related issue here. EDIT: the input lag issue is also on hardware but I don't know if its comparable to on PPSSPP .

anr2me commented 2 years ago

Don't forget the higher your latency the longer your input being delayed.

For example, assuming the game is using server-client netcode, where all players will be sending their state & action to the server/host, and then the server will broadcast this new state to all players so everyone have the same copy of the new state and be able to render the same thing on screen.

In this example i will assume that all players have 1 second latency, so when you press a button at midnight 00:00:00 and the action being sent to the server, it will arrived at the server 1 second later (which is at 00:00:01) and then the server broadcast this action/state to all other players(sometimes also send to the original sender), so the broadcast packet will arrived on all players 1 second later (which is at 00:00:02), and then all players will be able to render the action on screen, which will looked like it was delayed 2 second if you compare their screen with your(the original sender) screen. On some games, sometimes even the original sender will need to received this(his own) new state from the server before it can render the new state/action on screen (which is used to confirm that their new state/action have successfully received by the server), where they will noticed the delay on their own screen. If the packet lost on the way, it will need to be resend again, which will take another 1 second(latency) +a few(depends on the game) milliseconds (timeout for waiting the ACK or confirmation from the server), so it will get delayed much longer on a bad connection with high rate of lost packets. A connection with a high Lost Packet rate is much worse than a High Latency because there is additional delay to assume/confirms whether the packet was lost or not.

However, many adhoc games communicate directly player-to-player, so they're most-likely to broadcast their new state/action them self without going through a server/host, which mean the delay will be 1 second instead of 2 seconds (on 1 second latency)

If i'm not mistaken, even Xbox360 will drop connection that have more than 30ms latency on LAN play, so that's probably the recommended latency for games that was designed for LAN/Adhoc multiplayer, of course having 100ms latency probably won't make much difference as long you didn't compare the screen side-by-side, with 150+ latency you will start noticing the delay if you compare the screen side-by-side.

anr2me commented 2 years ago

I'll try to explain the difference between high latency but reliable vs. high latency and unreliable (high lost packet rate) connection with ASCII.

Assuming the latency is 300ms, but the game is using 200ms timeout to wait for ACK.

  0ms           100ms       200ms      300ms
Sender ---------->|---------->|----------> Server
   |                 Time to Resend Data     |
   |                                         |
   +<------------|<------------|<------------+ (ACK Reply)
 300ms          200ms         100ms         0ms
        (Time To Resend ACK)
  1. At the time reached 200ms (timeout), the game will assume that the packet is lost (because it didn't received the ACK yet) and need to resend the Data (where the resent Data starts from 0ms again, we will name this data DataCopy1)
  2. At the time reached 300ms, the Data arrived on Server and the Server reply with ACK (the DataCopy1 data that was resent is still at 100ms)
  3. At the time reached 400ms, the DataCopy1 data hit timeout again, and need to Resend (which will be DataCopy2 data, and so on)
  4. At the time reached 500ms, the ACK data also getting timeout on the Server side and need to Resend again (we will name this ACKCopy1)
  5. At the time reached 600ms, the ACK received by the Data Sender (which is Done sending), and the game can progress to the next step.

On a Reliable connection: It doesn't matter how many copy being sent, because the first sent Data/ACK didn't actually lost and will still arrived within 300ms since the connection is reliable, the receiver will only take one and discards the rest of the Copy/Duplicates.

On an Unreliable connection: The first sent Data/ACK may really be lost and need longer time (extra 200ms per copy) to arrived because the Copy is started from 0ms again, with a possibility to be lost again (resulting to a much longer time).

Note: Just because a connection have a reliable route to a public Adhoc Server (ie. pinging the server without getting request timed out), that doesn't mean the route between player-to-player will be reliable too.

ghost commented 2 years ago

Thanks for the long explanation but the delayed inputs / input lag also happens on localhost which is like 1ms lag? lol (wouldn't reported if it wasnt the case). Even Pursuit Force feels a bit floaty locally. Also a lot of games feel better input lag wise on psp it's only these 2 games that feel really off here.

BTW because PSP communicates P2P shouldn't the input lag be the same for all players? yet there a a difference in input lag between host and other players playing.

anr2me commented 2 years ago

P2P = peer-to-peer = player-to-player communication, unlike Server-Client where all the states centralized on the Server, the "Host" on a P2P aren't really managing the states (ie. decentralized)

So if Player1 ping to player2 = 200ms, but Player1 to Player3 ping = 100ms, and Player2 to Player3 = 300ms, of course each of them will experience different delays.

anr2me commented 2 years ago

Btw, are these games using GameMode? because i'm expecting a 5~10ms delay on GameMode, since we didn't sync the data immediately but using an interval basis on ScheduledEvent (i forgot whether the interval we use is 5ms or 10ms, but JPCSP is using 12ms which couldn't get a stable 60 FPS on some games if we use such a long interval/delay)

ghost commented 2 years ago

Neither are GameMode.. just AdhocMatching I guess. I think the issue with No Ken is more serious seeing as it completely fails to continue past the menus and into ingame lol.

anr2me commented 2 years ago

I've checked Hokuto no Ken: Raoh Gaiden, using simulated 250ms latency, the game will shows full speed 30/30 FPS. However, the rendering engine seems to be async with the game state synchronization engine, so the game will be moving like a slow motion. If you checked the countdown timer, each second in the countdown takes 3 seconds in real time.

Regarding the input, it wasn't delayed but more like being skipped/ignored (ie. if you pressed the jump button once and it got skipped your char won't jump no matter how long you waited), probably because the game is checking the input along with network sync, and the network being synced at a longer interval due to high latency, some buttons gets ignored/skipped. Pressing a jump button once often gets skipped with 250ms latency, so you'll need to press the button twice or more to give higher chance for the button to bump the same timing with sync time.

So the conclusion is that this game movements are affected by the latency thus not recommended to be played on high latency, probably only suitable for latency under 34ms (so the countdown timer won't fall behind with rendering time of 30 FPS which took 33 ms per frame), because that is how the netcode worked. Basically, if you're seeing the countdown timer moving slower than real time while the FPS is full speed 30/30, it means your latency is higher than recommended.

PS: This game uses regular PDP communication during gameplay, thus there is also a chance for packet to be lost on the way.

ghost commented 2 years ago

That was not the issue though... The issue was that the 2nd player inputs are delayed for the 1st player which caused a desync to happen and disconnect the game for everyone. It happens even on localhost with no simulated lag whatsoever. You can't reproduce it because the issue is not occuring on Windows whatsoever. I need to record this to show how it goes...

anr2me commented 2 years ago

Was it occurred only when no windows involved? (ie. happened on Android/Linux vs Android/Linux, but not happening on Android/Linux vs Windows)

If it doesn't occurred on Android vs Windows, i'm afraid i won't be able to debug it.

ghost commented 2 years ago

Then I am still not getting why it even happens only on my distro locally So weird.

anr2me commented 2 years ago

May be another inconsistent error code on Linux if it only happened on Linux/Android.

PS: i remembered there were a case where datagram packet got delayed for more than 10 seconds in the past (i forgot which game i was testing) and in happened on Windows's localhost. I thought localhost can have packet lost too back then, but apparently something is delaying the packet (may be my antivirus was scanning the packet for too long or something?), but because 10 seconds was too long for the game, the game ended showing disconnected message. However, this is a rare case which i haven't even bumped again (at least on Windows), but may be on other OS it could happen too, not sure why tho..

ghost commented 2 years ago

May be another inconsistent error code on Linux if it only happened on Linux/Android.

PS: i remembered there were a case where datagram packet got delayed for more than 10 seconds in the past (i forgot which game i was testing) and in happened on Windows's localhost. I thought localhost can have packet lost too back then, but apparently something is delaying the packet (may be my antivirus was scanning the packet for too long or something?), but because 10 seconds was too long for the game, the game ended showing disconnected message. However, this is a rare case which i haven't even bumped again (at least on Windows), but may be on other OS it could happen too, not sure why tho..

LOL just added videos for this at the same time you replied in the first post.

anr2me commented 2 years ago

Hmm.. looks like it's a video that's not supported by firefox O.o image I'll try chrome

anr2me commented 2 years ago

Those videos does looks like the packet got delayed for too long, do you have firewall on your linux? (most linux have firewall enabled by default)

Btw can you show me the ping result? ie. ping 127.0.0.2

ghost commented 2 years ago

The firewall is disabled by default on popos and the ping is like 0.090 ms (while concurrently trying to play) so it's not that. Just a really weird bug or issue.

anr2me commented 2 years ago

I remembered you're having an issue on linux similar to simulated 5ms++ latency on windows on another game (which were having timeout issue). May be that issue wasn't latency issue after all but packets can get delayed on linux localhost...

anr2me commented 2 years ago

Try using Debug channel on SCENET and compare the timestamp when one instance saying "Sent ... bytes" on sceNetAdhocPdpSent, with the timestamp on the other side that saying "Received ... bytes" on sceNetAdhocPdpRecv in order to find out whether it was delayed or not, and how long was the delay.

Both need to have the same packet size, and both need to be the first packet after sceNetAdhocPdpCreate (otherwise you will get confused if there are more than 1 packets with the same size, so we will only check the first packet after the socket created)

ghost commented 2 years ago

Try using Debug channel on SCENET and compare the timestamp when one instance saying "Sent ... bytes" on sceNetAdhocPdpSent, with the timestamp on the other side that saying "Received ... bytes" on sceNetAdhocPdpRecv in order to find out whether it was delayed or not, and how long was the delay.

Both need to have the same packet size, and both need to be the first packet after sceNetAdhocPdpCreate (otherwise you will get confused if there are more than 1 packets with the same size, so we will only check the first packet after the socket created)

Bad news... it doesn't show up in the logs at all. You can even see it doesn't show up when you see the 2nd video I sent... seems like the log stop working when i am in the menus but suddendly works after i get the disconnection message in the game ro something.

anr2me commented 2 years ago

Because it only logs Info channel in the video, so i thought you're not using Debug channel Then again, it might be because Debug channel got disabled on non-Windows, oh well, i'll check again when i had more free space to build PPSSPP for linux wsl later (not sure when tho)

But if you wanted to change the Log channel limitation, you can change these lines at Common/Log.h image This is the changed i did to enable up to verbose channel

ghost commented 2 years ago

I remembered you're having an issue on linux similar to simulated 5ms++ latency on windows on another game (which were having timeout issue). May be that issue wasn't latency issue after all but packets can get delayed on linux localhost...

Seems like something bigger then only linux localhost when it also happens on LAN though (or seems like it). I won't do that change in the source code because if you can't debug it there is just won't help much.

ghost commented 2 years ago

Like I said on Discord there is still the Gundam ZAFT input lag issue that increases the longer you play (every time you get a stutter for example). Only a 60 fps cheat would make it not noticeable for some reason (game is native 30 fps) at the expense of more FPS lag.

Same for Pursuit Force - Use a 60 FPS cheat to decrease the input lag but it only works on the racing mode , the on foot mode stays mostly the same which I guess is inherent to the game itself.

anr2me commented 2 years ago

It could be that packets get sent too fast (ie. faster than what the receiver could handle), because i've seen recv buffer gets filled up over time no mater how large it was (even on maximum allowed size), which could results to delays due to the most recent state is at the end of the queue.

It could be because we are using larger buffer than what the game is supposed to use to reduce stutters over high latency (due to too many dropped packets at the receiver end), we should at least use 1x buffer size on sent buffer which could delays sending process a little bit (ie. if sent buffer is full any attempt to send the next data will be blocked). But i'm still worried that some VPN (ie. RadminVPN) doesn't seems to like small buffer size, based on the test done at this old issue (where each builds being tested uses different buffer size) https://github.com/hrydgard/ppsspp/issues/13648#issuecomment-727948240

I also wondered if this too fast sending was also related to the enforced TCP_NODELAY=1(True) to improve response time, because PSP does have sceNetAdhocPtpFlush which supposed to be used to flush packets that still lingering on sent buffer due to Nagle algorithm, and with sent buffer size larger than what it supposed to be, packets could linger longer.

However, POSIX doesn't have such thing as "flush" API, and the only clue i found at stackoverflow was a trick that said can trigger flushing(i think it was from Linux users), by switching the socket's blocking mode forth and back (which is the one we use on sceNetAdhocPtpFlush implementation), but never knew whether this is really worked or not, especially on Windows.

anr2me commented 2 years ago

Btw, @MojoJojoDojo could you put the game list that gets delays even on localhost in your first post, so we won't loose it due to posts got sunk when this thread grown longer later.

unknownbrackets commented 2 years ago

It's no small amount of work, but ideally we should simulate the buffer size delays on the emulation side, irrespective of actual delays.

For example, maybe we need to have a 64 KB buffer, but the PSP had a 16 KB buffer. Ideally, we should keep track of the microsecond (in game time) at which the 64 KB buffer became full, and calculate how many microseconds until it should "correctly" drain.

Then we could continue to accept additional data, but block the PSP thread to simulate the delay it would have due to a full buffer. We'd need to do this anyway, because sendto() delaying even on the 64 KB buffer is not "visible" to the game, but should be (simulated.)

-[Unknown]

anr2me commented 2 years ago

I think JPCSP also use max buffer size (64k) but they seems to be managing the packets them self, at least on PDP datagrams. As i remembered i saw their code that read the whole datagram buffer (in a loop) and separate them into a list of messages, and i'm planning to do something like that too after moving AdhocSockets into a class, so we can calculate the correct recv_sb_cc (available data in recv buffer from PSP game perspective) value on sceNetAdhocGetPdpStat which need to be capped at the buffer size the game was using.

And yes, on most platforms, most of the time send/sendto return almost immediately because they only transfer the data to kernel/system, while the actual sending are done by the system/kernel.

Regarding the emulated/simulated delays, it will probably difficult since we don't know the accurate timings, may be measuring the time by sending to localhost using non-blocking mode for the HLE delay could be close enough.. While delays related to Nagle algorithm, i think there is a standard for this.

anr2me commented 2 years ago

Btw, how do you draws these lines? which part of the code does this? i wanted to create something like that too, to monitors available data on socket's recv buffers, so we can see whether buffers gets filled over time without the game capable to drains them. image

unknownbrackets commented 2 years ago

That's just drawn here: https://github.com/hrydgard/ppsspp/blob/cd7cea90a97bae84a6a1ef6b09e48f919a8be9c7/UI/EmuScreen.cpp#L1268

-[Unknown]

ghost commented 2 years ago

Btw, @MojoJojoDojo could you put the game list that gets delays even on localhost in your first post, so we won't loose it due to posts got sunk when this thread grown longer later.

I would add games that are on localhost with stimulated lag because my friend saw a relatively large difference between not using a 60 fps cheat and using one with medium/high latency (starting from 50ms up to 250ms) ,Also you cant rely on it because it's not the true latency between 2 or more PSPs.

In general you can notice it too on normal localhost if you are sensitive to this or you are using the on screen touch controls on PPSSPP / using it on mobile.

anr2me commented 2 years ago

Btw, @MojoJojoDojo could you put the game list that gets delays even on localhost in your first post, so we won't loose it due to posts got sunk when this thread grown longer later.

I would add games that are on localhost with stimulated lag because my friend saw a relatively large difference between not using a 60 fps cheat and using one with medium/high latency (starting from 50ms up to 250ms) ,Also you cant rely on it because it's not the true latency between 2 or more PSPs.

In general you can notice it too on normal localhost if you are sensitive to this or you are using the on screen touch controls on PPSSPP / using it on mobile.

The reason why i need i need to know whether it happened on localhost or not, because it shouldn't happen on localhost where the latency is minimal, which mean something when wrong.

Any latency above the normal Adhoc/LAN latency (which should be less than 30ms latency) could cause abnormalities, for example on Power Stone game if the latency is higher than 22ms the FPS will become higher than the normal 30 FPS, and the game will started to shows lags, because the game were designed for low latency such as Adhoc/LAN.

There are also more games that will shows abnormality on latency higher than 30ms, so you shouldn't use internet latency as reference(unless if it occurred on infrastructure mode), but use similar latency which the game were designed for.

anr2me commented 2 years ago

@MojoJojoDojo could you try my test branch here on linux https://github.com/ANR2ME/ppsspp/tree/socketbuffer And see whether it helped reducing the delays

ghost commented 2 years ago

@MojoJojoDojo could you try my test branch here on linux https://github.com/ANR2ME/ppsspp/tree/socketbuffer And see whether it helped reducing the delays

The issue with Hokuto No Ken still stands.

ghost commented 2 years ago

Tried to see what happens if you use Wine on Linux with multiple instances but it doesn't work there properly and disconnects after matchmaking immediately. Debug logs for both instances if it helps:

          00000000  50 50 53 53 50 50 00 00 00 00 00 00 00 00 00 00  PPSSPP..........
          00000010  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
          00000020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
          00000030  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
          00000040  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
          00000050  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
          00000060  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
          00000070  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
          00000080  00                                               .
24:08:038 main_thread  W[SCENET]: HLE\sceNetAdhoc.cpp:4594 UNTESTED sceNetAdhocMatchingStop(1) at 088165b0
24:08:046 main_thread  I[SCENET]: HLE\sceNetAdhoc.cpp:7741 InputLoop: End of InputLoop[1] Thread
24:08:055 main_thread  I[SCENET]: HLE\sceNetAdhoc.cpp:7456 EventLoop: End of EventLoop[1] Thread
24:08:056 main_thread  W[SCENET]: HLE\sceNetAdhoc.cpp:4669 UNTESTED sceNetAdhocMatchingDelete(1) at 088165bc
24:08:056 main_thread  W[SCENET]: HLE\sceNetAdhoc.cpp:4713 UNTESTED sceNetAdhocMatchingTerm() at 088165f8
24:08:056 main_thread  I[SCENET]: HLE\sceNetAdhoc.cpp:2481 Marked for Timedout Peer List (1)
24:08:056 main_thread  I[SCENET]: HLE\sceNetAdhoc.cpp:2517 00000000=sceNetAdhocctlDisconnect() at 08816040 [group=]
24:08:062 AdhocThread  D[SCENET]: HLE\sceNetAdhoc.cpp:5728 AdhocctlCallback: [ID=0][EVENT=2][Error=00000000]
24:08:072 idle0        D[SCENET]: HLE\sceNetAdhoc.cpp:417 Returning (WaitID: 3, error: 00000000) Result (00000000) of sceNetAdhocctl - Eve
: 0
24:08:073 AdhocThread  D[SCENET]: HLE\proAdhoc.cpp:1210 AfterAdhocMipsCall::run [ID=0][Event=2] [cbId: 0][retV0: 00000000]
24:08:523 main_thread  W[SCENET]: HLE\sceNetAdhoc.cpp:4713 UNTESTED sceNetAdhocMatchingTerm() at 088165f8
24:08:523 main_thread  I[SCENET]: HLE\sceNetAdhoc.cpp:2481 Marked for Timedout Peer List (1)
24:08:523 main_thread  I[SCENET]: HLE\sceNetAdhoc.cpp:2517 00000000=sceNetAdhocctlDisconnect() at 08816040 [group=]
24:08:524 AdhocThread  D[SCENET]: HLE\sceNetAdhoc.cpp:5728 AdhocctlCallback: [ID=0][EVENT=2][Error=00000000]
24:08:534 idle0        D[SCENET]: HLE\sceNetAdhoc.cpp:417 Returning (WaitID: 3, error: 00000000) Result (00000000) of sceNetAdhocctl - Eve
: 0
24:08:534 AdhocThread  D[SCENET]: HLE\proAdhoc.cpp:1210 AfterAdhocMipsCall::run [ID=0][Event=2] [cbId: 1][retV0: 00000000]
24:08:857 main_thread  W[SCENET]: HLE\sceNetAdhoc.cpp:4713 UNTESTED sceNetAdhocMatchingTerm() at 088165f8
24:08:857 main_thread  I[SCENET]: HLE\sceNetAdhoc.cpp:2528 sceNetAdhocctlDelHandler(0) at 08815f98
24:08:857 main_thread  I[SCENET]: HLE\sceNetAdhoc.cpp:2588 sceNetAdhocctlTerm() at 08815fa8
24:08:862 main_thread  I[SCENET]: HLE\proAdhoc.cpp:1759 FriendFinder: End of Friend Finder Thread
24:08:862 main_thread  I[SCENET]: HLE\sceNetAdhoc.cpp:2561 Cleared Peer List (1)
24:08:867 main_thread  I[SCENET]: HLE\sceNetAdhoc.cpp:3025 0=sceNetAdhocTerm()
24:08:877 main_thread  W[SCENET]: HLE\sceNet.cpp:602 sceNetTerm() at 08815fb8```

27:28:711 main_thread  D[SCENET]: HLE\sceNetAdhoc.cpp:1886 sceNetAdhocPdpRecv[1:
10777]: Received 128 bytes from 127.0.0.1:10777

27:28:744 main_thread  W[SCENET]: HLE\sceNetAdhoc.cpp:4713 UNTESTED sceNetAdhocM
atchingTerm() at 088165f8
27:28:745 main_thread  I[SCENET]: HLE\sceNetAdhoc.cpp:2194 sceNetAdhocPdpDelete(
1, 0) at 088189d8
27:28:745 main_thread  I[SCENET]: HLE\sceNetAdhoc.cpp:2481 Marked for Timedout P
eer List (1)
27:28:745 main_thread  I[SCENET]: HLE\sceNetAdhoc.cpp:2517 00000000=sceNetAdhocc
tlDisconnect() at 08816040 [group=ICE]
27:28:746 AdhocThread  D[SCENET]: HLE\sceNetAdhoc.cpp:5728 AdhocctlCallback: [ID
=0][EVENT=2][Error=00000000]
27:28:753 idle0        I[SCENET]: HLE\proAdhocServer.cpp:1007 AdhocServer: PPSSP
P (MAC: 3c:56:d0:4e:85:7f - IP: 127.0.0.1) left ULJM05404 group ICE
27:28:756 idle0        D[SCENET]: HLE\sceNetAdhoc.cpp:417 Returning (WaitID: 3, 
error: 00000000) Result (00000000) of sceNetAdhocctl - Event: 2, State: 0
27:28:756 AdhocThread  D[SCENET]: HLE\proAdhoc.cpp:1210 AfterAdhocMipsCall::run 
[ID=0][Event=2] [cbId: 1][retV0: 00000000]
27:32:615 HLE\proAdhocServer.cpp:697 I[SCENET]: AdhocServer: PPSSPP (MAC: 3c:56:
d0:4e:85:7f - IP: 127.0.0.1) stopped playing ULJM05404
27:32:616 HLE\proAdhocServer.cpp:1714 I[SCENET]: AdhocServer: Shutdown complete
27:32:616 HLE\proAdhocServer.cpp:1720 I[SCENET]: AdhocServer: End of AdhocServer
 Thread
27:32:618 HLE\proAdhoc.cpp:1759 I[SCENET]: FriendFinder: End of Friend Finder Th
read
27:32:618 HLE\sceNetAdhoc.cpp:2561 I[SCENET]: Cleared Peer List (1)
27:32:817 Util\PortManager.cpp:507 I[SCENET]: UPnPService: End of UPnPService Th
read
012c:fixme:kernelbase:AppPolicyGetProcessTerminationMethod FFFFFFFFFFFFFFFA, 000000000011FD70
anr2me commented 2 years ago

Hmm that's odd, one side is connected to group=ICE while the other doesn't have a group name

ghost commented 2 years ago

Well I got Hokuto No Ken to work with the verbose ppsspp build and got this from one instance:

4:51:446 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1887 sceNetAdhocPdpRecv[1:10777]: Received 128 bytes from 127.0.0.1:10777

24:51:447 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff460, 128, 0, 0) at 08817f1c
24:51:447 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:480 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1887 sceNetAdhocPdpRecv[1:10777]: Received 128 bytes from 127.0.0.1:10777

24:51:480 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff49c, 128, 0, 0) at 08817f1c
24:51:480 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:480 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff460, 128, 0, 0) at 08817f1c
24:51:480 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:513 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1887 sceNetAdhocPdpRecv[1:10777]: Received 128 bytes from 127.0.0.1:10777

24:51:513 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff49c, 128, 0, 0) at 08817f1c
24:51:513 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:513 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff460, 128, 0, 0) at 08817f1c
24:51:514 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:547 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1887 sceNetAdhocPdpRecv[1:10777]: Received 128 bytes from 127.0.0.1:10777

24:51:547 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff460, 128, 0, 0) at 08817f1c
24:51:547 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:580 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1887 sceNetAdhocPdpRecv[1:10777]: Received 128 bytes from 127.0.0.1:10777

24:51:580 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff49c, 128, 0, 0) at 08817f1c
24:51:580 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:581 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff460, 128, 0, 0) at 08817f1c
24:51:581 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:613 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1887 sceNetAdhocPdpRecv[1:10777]: Received 128 bytes from 127.0.0.1:10777

24:51:614 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff49c, 128, 0, 0) at 08817f1c
24:51:614 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:614 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff460, 128, 0, 0) at 08817f1c
24:51:614 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:647 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1887 sceNetAdhocPdpRecv[1:10777]: Received 128 bytes from 127.0.0.1:10777

24:51:648 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff460, 128, 0, 0) at 08817f1c
24:51:648 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:680 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1887 sceNetAdhocPdpRecv[1:10777]: Received 128 bytes from 127.0.0.1:10777

24:51:680 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff49c, 128, 0, 0) at 08817f1c
24:51:680 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:681 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff460, 128, 0, 0) at 08817f1c
24:51:681 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:714 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1887 sceNetAdhocPdpRecv[1:10777]: Received 128 bytes from 127.0.0.1:10777

24:51:714 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff460, 128, 0, 0) at 08817f1c
24:51:714 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:747 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1887 sceNetAdhocPdpRecv[1:10777]: Received 128 bytes from 127.0.0.1:10777

24:51:747 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff49c, 128, 0, 0) at 08817f1c
24:51:747 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:747 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff460, 128, 0, 0) at 08817f1c
24:51:747 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:780 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1887 sceNetAdhocPdpRecv[1:10777]: Received 128 bytes from 127.0.0.1:10777

24:51:780 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff49c, 128, 0, 0) at 08817f1c
24:51:780 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:781 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff460, 128, 0, 0) at 08817f1c
24:51:781 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:813 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1887 sceNetAdhocPdpRecv[1:10777]: Received 128 bytes from 127.0.0.1:10777

24:51:814 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff460, 128, 0, 0) at 08817f1c
24:51:814 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:847 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1887 sceNetAdhocPdpRecv[1:10777]: Received 128 bytes from 127.0.0.1:10777

24:51:847 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1520 sceNetAdhocPdpSend(1, c8:74:af:af:59:e0, 777, 0x7f1809fff49c, 128, 0, 0) at 08817f1c
24:51:847 main_thread  D[SCENET]: HLE/sceNetAdhoc.cpp:1609 sceNetAdhocPdpSend[1:10777]: Sent 128 bytes to 127.0.0.1:10777

24:51:881 main_thread  W[SCENET]: HLE/sceNetAdhoc.cpp:4714 UNTESTED sceNetAdhocMatchingTerm() at 088165f8
24:51:881 main_thread  I[SCENET]: HLE/sceNetAdhoc.cpp:2195 sceNetAdhocPdpDelete(1, 0) at 088189d8
24:51:881 main_thread  I[SCENET]: HLE/sceNetAdhoc.cpp:2482 Marked for Timedout Peer List (1)
24:51:881 main_thread  I[SCENET]: HLE/sceNetAdhoc.cpp:2518 00000000=sceNetAdhocctlDisconnect() at 08816040 [group=ICE]
24:51:897 AdhocThread  D[SCENET]: HLE/sceNetAdhoc.cpp:5729 AdhocctlCallback: [ID=0][EVENT=2][Error=00000000]
24:51:898 idle0        D[SCENET]: HLE/sceNetAdhoc.cpp:418 Returning (WaitID: 3, error: 00000000) Result (00000000) of sceNetAdhocctl - Event: 2, State: 0
24:51:898 AdhocThread  D[SCENET]: HLE/proAdhoc.cpp:1212 AfterAdhocMipsCall::run [ID=0][Event=2] [cbId: 0][retV0: 00000000]

THe other instance got the same exact log so it won't be helpful to post it.

anr2me commented 2 years ago

The Logs didn't shows anything strange, they're able to communicate successfully without error on PdpSend and PdpRecv, so i can't really tell what's wrong with it.

ghost commented 2 years ago

Reminder that the Gundam input lag issue is noticeable even on localhost (maybe even more with 4 player instances). The issue is on all platforms this time and the input lag increases until it stays fixated. One player may get no input lag though. This is best noticed with the on screen controls on.

ghost commented 2 years ago

Xiaolin Showdown got input lag issues in the lobby menu and ingame too. Pretty big input lag as well and noticeable. Game also seems to desync Android to PC (?) prolly math problem this time.

ghost commented 2 years ago

Obscure The Aftermath got input lag for the player connected to the host. Same for The Warriors. locally phone to PC

anr2me commented 2 years ago

It would be nice if you can find out which version was the last working version, at least which stable version if you didn't want to test all of those git builds (even though there aren't many PR related to networking)

ghost commented 2 years ago

It would be nice if you can find out which version was the last working version, at least which stable version if you didn't want to test all of those git builds (even though there aren't many PR related to networking)

I think its the ones you changed the buffer size because they are inconsistent between host and clients I remember you said. It makes the most sense to me. But I will check previous stable versions later maybe.

anr2me commented 2 years ago

Buffer sizes are set by the game, it's normal for games to use different size for the host and client (since the host need to handle packets from many clients), but it depends on how the netcode worked.

Most adhoc games are p2p which isn't centralized to a certain player, as they communicate directly player to player (eg. each player will sent their packet to everyone and receives packet from everyone), but there might be games that use the same code base as their infrastructure mode and have a client-server netcode too for adhoc.

ghost commented 2 years ago

I meant what you said here: https://github.com/hrydgard/ppsspp/issues/15336#issuecomment-1028570125

BTW if you want a game you can notice input lag easily then you got Xiaolin Showdown as it happens right in the multiplayer/adhoc menu too as well lol.

Too bad there is no 1.10.3 build with localhost I could use to find out what happens there. But most of the games that exhibit input lag are fixed games from after 1.10.3 anyway.

anr2me commented 2 years ago

I meant what you said here: #15336 (comment)

BTW if you want a game you can notice input lag easily then you got Xiaolin Showdown as it happens right in the multiplayer/adhoc menu too as well lol.

Too bad there is no 1.10.3 build with localhost I could use to find out what happens there. But most of the games that exhibit input lag are fixed games from after 1.10.3 anyway.

you meant during this screen? image input response time feels normal for a 20 FPS game tho, but if the input delay worsen over time and it happened on linux, probably the same issue with "no Ken" game, caused by socket buffer filled up and didn't get received fast enough (due to FIONREAD not returning an accurate number on linux)

ghost commented 2 years ago

I meant what you said here: #15336 (comment) BTW if you want a game you can notice input lag easily then you got Xiaolin Showdown as it happens right in the multiplayer/adhoc menu too as well lol. Too bad there is no 1.10.3 build with localhost I could use to find out what happens there. But most of the games that exhibit input lag are fixed games from after 1.10.3 anyway.

you meant during this screen? image input response time feels normal for a 20 FPS game tho, but if the input delay worsen over time and it happened on linux, probably the same issue with "no Ken" game, caused by socket buffer filled up and didn't get received fast enough (due to FIONREAD not returning an accurate number on linux)

I meant the menu right after you press "Multiplayer" but that works too. I tried to see whats the difference between the input lag on my PSP and PPSSPP and PSP is smidge faster in reponse time then PPSSPP (of course with Force Real Clock sync enabled).

Also input lag that increases over time doesnt apply to the Gundam Z.A.F.T game I mentioned here before when only talking about Linux. Seems to be universal between all OSes for that one. I am also not the only one that mentioned it (someone mentioned it on PPSSPP forums ages ago).

Anyway updated the list of games again on the first post (note: for some of them you can notice the input lag only on the 2nd player).

anr2me commented 2 years ago

If you're talking about moving the highlight between HOST and JOIN, pressing up/down to change the highlight doesn't even use/need networking function, it's purely UI of the game, and the response time is quite normal for 20 FPS (but may be PPSSPP's input doesn't worked well on 20 FPS games for the delay to be noticeable compared to real PSP), unlike single player that runs at 60 FPS, you may want to try using 60 FPS cheat (if any) to see whether it worked as smooth as single player or not tho. image

PS: you can change the opcode at 0896F238 to b 0x0896F268 to get 60 FPS on this menu. image

ghost commented 2 years ago

Forgot about Blazblue games also having input lag issues on multiplayer. I checked console vs PPSSPP long ago and PPSSPP got worse input lag here (in comparison anyway) Also someone on discord reported about it recently: https://discord.com/channels/293316141479362560/293316141479362560/1029668114939514880 Added the games to the list.

anr2me commented 2 years ago

Forgot about Blazblue games also having input lag issues on multiplayer. I checked console vs PPSSPP long ago and PPSSPP got worse input lag here (in comparison anyway) Also someone on discord reported about it recently: https://discord.com/channels/293316141479362560/293316141479362560/1029668114939514880 Added the games to the list.

And what was the FPS of this game during multiplayer? hopefully not another 20 FPS game, which PPSSPP input seems to have slower response on 20 fps (based on Xiaolin Showdown menu that doesn't even use networking function yet)

ghost commented 2 years ago

It's a 60 fps game this time lol. Of course its more noticeable when you use the touch button controls then using normal ones. You can enable touch controls even on PC in Controls settings.

ghost commented 2 years ago

It would be nice if you can find out which version was the last working version, at least which stable version if you didn't want to test all of those git builds (even though there aren't many PR related to networking)

Someone noticed a change in input lag between versions here: https://discord.com/channels/293316141479362560/480103153803460614/918050823517392927 Didn't check if the commits are ad-hoc related though.