[Bug]: Network-Game synchronization failed (desync error)

[ferares]

Version of OpenTTD

Tested both 13.4 (Slowpace 1.3.1) & 12.2 (Slowpace 1.3.1)

Expected result

When running a dedicated server on Linux, Windows clients should be able to connect and play and vice-versa.

Actual result

When hosting a dedicated server on Linux, Windows clients will get a desync error and get kicked out. This also happens on Windows servers with Linux clients so it seems like its some sort of cross-platform issue. The desync error seems to happen as soon as there are vehicles moving around. I've tried collecting more debugging data by setting debug_level desync=4 but it does not seem to work... or maybe I'm doing that wrong. Let me know how can I provide more data and I will

Steps to reproduce

1. Start a dedicated server on Linux or Windows
2. Connect to that server using Windows or Linux (not the same OS as the server)
3. Build a depot of any kind, buy a new vehicle and make it go so it starts moving around
4. The server should kick you with a message saying "Network-Game synchronization failed"
5. On the server logs you should see "dbg: [net] '[username]' reported an error and is closing its connection: desync error"

[kaomoneus]

Wow! Currently I'm a bit far from my laptop, probably will reach it at Jan 8. Perhaps I could try to fix it analytically. Once I had desyncs, need to check how I fixed it last time and what was the issue.

What about Linux-Linux pairs and Windows-Windows? Is that ok?

[kaomoneus]

Found desync issue i fixed last time

https://github.com/kaomoneus/OpenTTD/commit/f42985b9d9c5c575133a878ba2ef7a8bc1ebb111

[ferares]

That's great, do you remember what you did to debug that?

Linux-Linux and Linux-OSX work just fine

I've tested all the combinations I'm able to test again and these were the results:

• Server - Client

• [X] Linux - Linux

• [X] Linux - OSX

• [ ] Linux - Windows

• [X] Windows - Linux

• [X] Windows - OSX

• [X] Windows - Windows

• [X] OSX - Windows

• [X] OSX - Linux

The only one that failed this time was Linux server with Windows clients (Previously Windows - Linux failed on me but now it is not failing).

Again the server remains up and working fine but Windows clients get kicked out due to desync errors.

The error only happens if there are vehicles moving, if I manually stop every vehicle on the map the Windows clients have no issues, but when a vehicle starts moving again the desync error occurs again and they get kicked out.

[kaomoneus]

I'll try to check this. Are you sure that it's not affected by towns growing?

[ferares]

It looks pretty tied to vehicles moving, here's a video of the bug:

https://drive.google.com/file/d/1InVuAciJ74lHGLLspkv6KnORxz0HkGDV/view?usp=sharing

The problem seems to happen regardless of what pace I set for the game, on the video is the original game pace but I've tried with one game year per day and the issue is the same.

As you can see, shortly after the ship starts moving around the Linux server kicks the Windows player out, if the player rejoins it gets kicked out again.

You can also see on the video that the time windows players can stay on the server before being kicked out is not always the same. It varies a bit but it's always a few seconds.

I've tried with buses, planes and trains too they all have the same effect.

If I join as a Linux client and stop all the moving vehicles on the map then the Windows player can join and stay in without being kicked out.

[kaomoneus]

Ok, thank you! Let's suppose that the cause is a vehicle-breakage mech. Do you know how to disable it? Could try it again with that mech disabled?

[ferares]

Just did a quick test disabling vehicle-breakage and it seems like that's it! there are no desync errors now.

I'll do some more exhaustive tests later but it looks like the issue is related to that.

[kaomoneus]

Wow! That's great, thank you 👍 It narrows possible place to fix quite a lot.

[kaomoneus]

@ferares , so how it goes? Did you had any chance to run a long test with vehicle breakdown disabled?

[ferares]

@kaomoneus yes, sorry, we played for several hours with some friends on a Linux server with breakdowns disabled using a mac, a Linux and a windows client and there were no issues 🙌

[kaomoneus]

Good! Hope you enjoyed) The problem is like 10 strings of code, unless I can't see any problem there, haha. Let me meditate on this a little bit more...

[ferares]

Yeah, we're enjoying this mod a lot, we like letting the server running while no one is on and not coming back to it to find 30 years have gone past lol

You might already know this info but just in case we found this on the wiki https://wiki.openttd.org/en/Manual/Multiplayer#desync-error that might help

We're happy playing without breakdowns so please don't feel pressured

Were you able to reproduce the error on your end?

[kaomoneus]

Were you able to reproduce the error on your end?

At current moment I have no laptop, all I can do is to read source from mobile, I also has some primitive linux env on my phone and primitive compiler in my head, haha. So while my wife taking photos of every tree and every corner of surroundings I'm checking memorized code and trying to reproduce it in my mind :-D

[ferares]

😅 that's alright, like I said there's no rush. Enjoy with your wife and let me know if you need me to set up a server/client or get some logs when you're back on your laptop. And happy new year! 🎊

[kaomoneus]

@ferares pay attention to towns growing by the way, as I can see, it may grow faster than usual..

[kaomoneus]

😅 that's alright, like I said there's no rush. Enjoy with your wife and let me know if you need me to set up a server/client or get some logs when you're back on your laptop. And happy new year! 🎊

Thanks, same to you :-)

[kaomoneus]

@ferares , could you please attach saved game? Thank you!

[ferares]

Sure, here it is @kaomoneus Showtime.sav.zip

[kaomoneus]

Sure, here it is @kaomoneus Showtime.sav.zip

Thanks! will check it!

[kaomoneus]

@ferares , looks that I've fixed issue. Please try out these builds: windows-gbc5d linux-gbc5d macos-gbc5d - this might have color issues, seems like github CI uses wrong endian settings, if you also observe them, let me know I'll rebuild it on my old mac.

I also have launched a linux server with your game. So you can just connect to it and test. You can add it by invite code +pWyzaCE

[ferares]

@kaomoneus it works! sorry I took so long to test it out but I can confirm the game is working as it should now.

I've tested both a Linux server and a Windows one with Linux and Windows clients and vehicle breakdowns active and everything seems to be working fine, at least I'm not getting an instant "desync error".

Thank you for all the work you've put in and this awesome mod!

[kaomoneus]

@ferares , great! It's now time to add new release. So far, feel free to use my server!

[ferares]

@kaomoneus we decided to start a server without vehicle breakdowns back when you discover that was the culprit, so we already have a server going but thanks for the offer 😊

[kaomoneus]

@ferares hi! I is slowpace experience?