Allow updating remote parameters by network-syncing certain OSC messages

[seanedwards]

*What's the idea?** This is a proposal for network synced OSC messages. This proposal is closely related to the Allow driving local parameters with OSC proposal. I'm submitting this as its own proposal to avoid derailing that one with a different topic, but I'm happy if someone wants to merge them as well.

Certain OSC messages themselves could be sent (and rate limited) as events over the network. Even though the animator parameters themselves would only be local, everyone's local copy could update at roughly the same time when an OSC message is broadcast. This is similar in some ways to the difference between Udon variables and events.

Perhaps the OSC json config can have a property specifying whether messages should be networked.

        {
            "name" : "Face",
            "input" : {
                "address" : "/avatar/parameters/Face",
                "type" : "Int",
                                "synced": true // Only makes sense for inputs to be synced
            },
            "output" : {
                "address" : "/avatar/parameters/Face",
                "type" : "Int"
            }
        },

By having an inventory of the parameters that can be synced, the network protocol could be made quite efficient as well. Rather than transmitting a full parameter name on every update, the client could do something like:

• When another player joins, send them our mapping of parameter names to binary IDs.
• As the game runs, send packets of the form 0x00 0xFF (set the first networked OSC slot (0x00), which according to our mapping above is the Face parameter, to 127).

This example protocol could give players up to 256 possible OSC parameters that could be networked, at 2 bytes per message. (Maybe 16-bit numbers are affordable? 🙏 )

A rate limit could be built on top of this, where rather than limiting the number of parameters, we limit the total bitrate of OSC updates: "You get to transmit up to 512 bytes/sec in OSC updates before messages start getting dropped."

Is there another way? The only other way to network avatar data is through animator parameters, which are very limited (for good reason!) and have semantics that are very useful for some things (like reliable, atomic updates of multiple parameters) but not needed for everything.

[bdunderscore]

I'll note that if this is implemented, it would be useful to also consider a state behavior that could drive transmitting a synced avatar message without going through OSC.

[bdunderscore]

Now that Contacts are public: Triggering an RPC-like message from remote users to the avatar owner would be a nice feature for AVD as well, and would fit in nicely with similar messages in the other direction from OSC.