speed tests

[mithrendal]

Hi @mikaelho

i made a small program to speed test the different communication modes of multiplier connectivity via your API. For the test it send small text string 'ping' over to another peer that peer answers the incoming message with a 'reping'. The program does this 1000 times to get a reasonable average. So here are the results for the different modes:

11.85ms send method unreliable 11.94ms send method reliable 6.19ms stream method

there is effective no difference in latency whether using send method mode reliable or unreliable. When you run the test again it is unpredictable which one has lower latency. But stream makes really the difference.

When I measured it last month or so, I saw unstable latency values with the send method. But it turned out that the latency came from the GUI, (maybe garbage collector?) sometimes I saw values of 240 or so.

Now I changed the benchmark program that it takes 1000 probes automatically without GUI intervention. And the 1000 probes are very close to each other I see no statistical outliers anymore.

So yeah. You brought latency effectively down a lot with streams. So great!!!

Before I saw these good values for streams, I had to remove some debug code in multipeer.py. You still have some debug-log code in method 'streamhandleEvent' where you write to a file named 'l'. I also disabled global logging by inserting a 'pass' into the method 'log_this'. Maybe you still would like to do some housekeeping ? Or should I do it and send you the pull request ?

Thanks a lot for this addition !!! This is soo great !

[mikaelho]

Thank you for these tests and for sharing the results. I included them in the documentation in commit d839ab10f675a33339e1b59fda5cdf172caaa9a4, which also includes some housekeeping. Next I will check whether the lightcycle game works any better without the debugging code.

Leaving this issue open for the moment.

[mikaelho]

Quick check shows that lightcycle still has issues - ”master” runs smoothly but ”clients” are choppy. Performance measurements indicate that the flaw is in lightcycle design and not in the multipeer itself.

[mithrendal]

For me the lightcycle game is not soo bad. I played it with two devices. (In fact it is a lot of fun, played it with my kids dozens of times. They always want a revenge. Quite addictive gameplay ;-) ...) How do you know which device is master? Maybe I was always on the master. I will try to extract the benchmark program from my main project into a standalone file and put it here. So you can better test out the latency.

[mikaelho]

The problem is more visible with three or more players. But now that you have ”proven” that latency is probably not the issue, I will change it to a true peer setup.

By the way, there is also a single-player practice mode, which you can activate on line 593 by changing the mode to Game instead of PeerGame, and giving the number of robots to race against.

I was thinking about a ”coding challenge” for the Pythonista forum, where people could submit their robot AIs.

[mithrendal]

Don't know whether the "prove" is still true for more than two peers. I will have to try to do a testing setup with more than two peers.

"coding challenge" Wow that would be nice. Reminds me on the famous robocode project where you had to code tank units. Loved that 18 years ago !

[mithrendal]

@mikaelho I just made a pr. I separated the code from my stuff into a single file called speedtest.py. But I was astonished that with the separated code the values for send get worse around 20ms. Back to my own code around 11ms again. I went into the kitchen, better ;-) 13 ms. Stream is in speedtest.py now around 8ms. Back to my own game stuff stream is 6ms. Will do some checkup on this issue tomorrow. In my opinion it should be the same in both programs.