NexusSocial / nexus-vr

Monorepo for most code
Apache License 2.0
18 stars 7 forks source link

Client-Side Scripting #82

Open CorneliusCornbread opened 6 months ago

CorneliusCornbread commented 6 months ago

Summary

To empower developers of Nexus to create advanced applications the ability to have a Nexus server have clients execute custom behavior is necessary. One of the solutions that have been proposed to this problem is requiring Nexus clients to support WASM. This however poses a few concerns, mainly with performance, and possibly security as well.

Philosophy of Design

The majority of applications we see in social VR apps today make heavy use of some form of client-side scripting. VRChat implements Udon and Resonite implements ProtoFlux. Despite some disagreement, there is a general sentiment of not creating something from scratch, unlike what VRChat and Resonite, be that an existing scripting language or WASM. I cannot speak to the reasoning behind using a specific scripting language (like maybe JavaScript or LUA), but in terms of the benefits of say WASM it would allow developers to essentially use whatever tools they choose. Any given language, so long as it can be compiled to WASM, would be supported that way.

Drawbacks/Blockers

The main concern of scripting would be performance, a common worry is when the execution of these scripts happens. Should scripts be executed every frame? Should we instead use some sort of event-driven system? What about UI, what's the most reasonable way we can create an API for creating application-specific UI that is performant? Etc. This is an ongoing discussion and is going to require decisions to be made specifically on the level of how this should be implemented. Moreover is the discussion of API, which such a scripting system is going to require.

CorneliusCornbread commented 6 months ago

On the topic of WASM, I think this is a good resource to understand some of the limitations of the language. https://github.com/WebAssembly/design/blob/390bab47efdb76b600371bcef1ec0ea374aa8c43/Nondeterminism.md

Though WASM is actually surprisingly deterministic despite these limitations.