1.8.0 NetworkVariable mismatch between Server & Client

[aidanhorton]

Description

It's possible that it's a bug in my code rather than the other way round - but basically I'm logging a network variable in a client, and then logging it in the server host client, and getting different results.

I was using NGO 1.7.1, and updated to 1.8.0, and now NetworkVariables seem to have synchronization issues. It's a complex project so I'm still working out if the bug is limited in scope.

This only happens if I switch scenes. So say I start in my main scene, both clients connect to that scene - the synchronization is fine. But if I start in a lobby scene for example, and then migrate over to the new scene (new players are created for each client in each scene, there is no player or network object migration) - then I get this issue where the NetworkVariables don't match.

Reproduce Steps

Can't offer exact steps at this precise time. The TLDR is; I have a NetworkVariable called 'State' which holds an enum. I have an update loop that debug logs the value of this network variable when I press a key. If I press it on the client and the server (and I know that the value hasn't changed between the presses), then it's logging different values. The host client and client are definitely connected as the players can see each other.

I do have a fairly complex inheritance structure, but that shouldn't affect this specific logging.

Actual Outcome

Network Variable mismatch between host client and regular client.

Environment

• OS: Windows 11
• Unity Version: 2023.2.0b17
• Netcode Version: 1.8.0

Let me know if any further information is required. I'll update this bug as I find out more information.

[NoelStephensUnity]

Hi @aidanhorton,

Could you provide more details on the NetworkObject with the NetworkBehaviour containing the NetworkVariables not synchronizing properly? (i.e. is it in-scene placed, when is is spawned, is it a component on the players, etc.)

Without knowing that, the only recent update (between v1.7.1 and v1.8.0) that "might" impact this would be that any CreateObjectMessages received by a client that is in the middle of scene synchronization (whether a newly joined client or during scene transition) will get deferred until the client has finished synchronizing. So, if any changes were applied to a NetworkVariable (on the server side) of a newly spawned NetworkObject that was spawned while the client was still synchronizing then if your NetworkManager's Spawn Timeout was still set to a legacy value (i.e. 1 second) it could theoretically drop any NetworkVariableDeltaMessages for any NetworkObjects that were created on the server side after it had finished loading the scene (and the server updated any NetworkVariables on it).

So, additional information about the object with the NetworkVariables will definitely help but also you might check that Spawn Timeout value and if it is still 1 second then adjust it to like 10 seconds (just to give plenty of wiggle room depending upon the size of the scene being loaded).

If that does nothing or it was already set to a reasonably high value, then definitely providing a bit more information on the NetworkObject with the NetworkVariables will help us figure out what could be the cause.

[aidanhorton]

Hi @NoelStephensUnity !

So the NetworkObject which I can confirm I'm getting this issue on (it may be happening in other cases, but this is the most visible), is an in-scene placed GameObject - and the NetworkBehaviour is on this object.

I believe the value is being modified during or immediately after synchronization - I've implemented by own method that gets triggered after all OnNetworkSpawn methods have been run, and this gets called either OnSceneLoadComplete, or OnSynchronizeComplete. That's when the network variable is getting modified on the server.

I noticed below that I was getting a lot of warnings (which I never knew exactly how to interpret) in 1.8.0, where I wasn't on 1.7.1.

So yes, the client is still synchronizing itself when this variable has already changed.

Interesting implementation - is the 1 second timeout meant to be higher than 1 second, you mention its a legacy value? I haven't manually set it to any value in my code - but when I set it to 10 seconds for example, it worked just fine. What's the negative to me setting this to 60 seconds for example so that slow loading clients would never see this issue?

[aidanhorton]

I've just continued testing 1.8.0, and I get a bunch of other issues too - even with the timeout set to 10 seconds.

My main issue is that NetworkRigidbodies seem to be in random locations (although I don't have anything that's solely a network transform). They're in a certain position in the server (the correct position), but on the client they're entirely incorrect - floating in the sky in most cases in completely the wrong position...

It's weird they all seem to be converging towards 0,0,0, and if they're parented to something, they're correct relative to each other, but they seem to be within a 10x10 radius of 0,0,0 instead!

[NoelStephensUnity]

Regarding the SpawnTimeout value, this basically allows you to adjust how long a client will defer messages while synchronizing in the event you are loading larger scenes (or the like). This value can be adjusted during runtime, so you could have the server send a message to clients, prior to loading scenes, that adjusts their SpawnTimeout values or just set it to a large value. The downside to a large value is that the client will queue up messages for up to that period of time (if a scene or scenes or a newly joining client takes that long to synchronize)... so the downside is just memory consumed while the messages queue up. They are dequeued and processed in the order they were received.

Regarding your floating NetworkRigidBodies, I am assuming they include a NetworkTransform? Just to make sure, are the NetworkObject's "Synchronize Transform" value enabled? If so, what are the NetworkTransform settings and are these in-scene placed or dynamically spawned objects?

[aidanhorton]

@NoelStephensUnity Cheers for the info, makes sense! Is it that messages get deferred for 10 seconds (if set to 10 seconds), then they load in only after 10 seconds - even if sync only takes 3? Or if sync takes 3 seconds, it'll defer for a max of 10 seconds, but only 3 seconds in this case?

Apologies, I wrote that issue quickly, I don't know why I didn't include all that info! They all include a NetworkTransform. They all worked in 1.7.1 & earlier. Synchronize Transform is enabled. Network Transform settings are default from previous versions (I'll include an image below, the custom version of net transform is basically client network transform, no other logic):

These are in-scene placed. This happens whether loading into the scene from another one, or whether starting straight off in this scene where I see the issue. If dynamically placed, they sync immediately, I assume due to the positional/velocity updates being sent as they have gravity. As soon as the rigidbody is moved, then the position updates on all clients to the correct position. Again, it seems to be an initial synchronization issue! They are initially owned by the server - meaning even though they have gravity enabled, they are floating on other clients (since they're kinematic while not owner by the local client).

I've also now tested in 2023.2.9f1 (stable) and still get the same issue.

[aidanhorton]

Any ideas on this one? I'm working on a refactor of our networking code and could really do with the new NetworkManager events and RPC attributes, so I'd like to get this solved as soon as I can! Thanks for the help so far.

[travis-tap-3d]

FWIW I'm running into the same issue after upgrading from 1.7.1 to 1.8.1-- i.e. my NetworkTransform's are no longer syncing the initial transform properly. If I toggle the local space option on and off it fixes itself so it appears to be the initial transform setup thats the issue. Works fine in 1.7.1. Adjusting spawn timout doesn't seem to make any difference.

[NoelStephensUnity]

Ok, So I have a simplified project that we can use as a point of reference since this seems to be a more complex setup type scenario. Whether server or client authoritative, anything in-scene placed will default to being owned by the server when loading a level. As such, I am going to speculate that somewhere in the midst of loading the scene or when the scene loading is complete you are transferring ownership to another client?

@aidanhorton I will need you to look over the attached project and make modifications (if possible) in order to get it to more closely reflect what you have setup on your side. This will help me narrow down what issue you are experiencing.

@travis-tap-3d I am assuming you have the same kind of issue (but want to make sure since there still seems to be some specifics needed)? If you could do the same thing (i.e. modify the attached project) to more closely represent what you are experiencing it would help me narrow down where the synchronization issue is occurring.

For both of you, when you are done making modifications and want to submit if you could do the following it will greatly reduce the zip file size:

Only include the following folders:

• Assets
• Packages
• ProjectSettings

In project settings you only really need to include:

• EditorBuildSettings.asset
• ProjectSettings.asset
• ProjectVersion.txt

When I get the project updates from each of you, I will be able to take a look and narrow down what could be the issue and determine if there is a minor adjustment you could make to resolve the issue or if it is a bug I will create a PR to resolve it and post instructions on how you can modify your manifest file to include those updates until the fix is included in the next update.

The Project To Modify

InSceneRigidBody.zip

[aidanhorton]

@NoelStephensUnity can confirm that my issue is related to this one here: https://github.com/Unity-Technologies/com.unity.netcode.gameobjects/issues/2888

[NoelStephensUnity]

@aidanhorton Did disabling the auto parent sync on the child NetworkObject resolve your issue?

[aidanhorton]

@NoelStephensUnity that does provide a fix for the issue - but aswell modifying the NGO with the fix provided in this issue thread also resolves it https://github.com/Unity-Technologies/com.unity.netcode.gameobjects/issues/2888