write an overview/guide how to allow for community content on community servers without sandboxing

Describe the project you are working on

Any multiplayer game with community content and community servers, such as: Quake, Minecraft, Unvanquished and even beyond.

What lies beyond? You could for example create a "Godot Metaverse", where people connect to custom environments that offer totally new ways of interactions. Those environments could be accessed just like websites from custom servers and they are basically just custom scenes with custom scripts and custom assets.

I also want to note that creating multiplayer games without community servers is bad practice, often frowned upon, and always a huge disadvantage to the community around your game. Some games are very simple or don't need community content. But for a lot of "mid-sized" games it is one of the most important aspects of the game.

Describe the problem or limitation you are having in your project

Godot workflows are scene based. A map is a scene, a monster is a scene, a player is a scene, a menu is a scene, everything is a scene. Game code is always intricately interwoven with those scenes in a sensible object-oriented manner as scripts. It is not true that game code defines objects and interactions in the game. Scenes define objects which define arbitrary interactions and arbitrary game code.

Scripts are not sandboxed and thus community content is basically a gateway to arbitrary code execution. This means that anyone can turn your game into a trojan/virus for any player who loads community content, as community content is always untrusted, and basically anyone can get someone else to load it most of the time even from within the game itself. So you clearly can't do it: a total no-go. But there are of course also some ways, some of the time around that. Here is where the problem starts: I have heard mostly that community content is not possible in Godot over the years, very simply due to lack of sandboxing. But then also people claimed it is. And some also said that Godot has sandboxing and has mod support - in some sense at least. Again and again you have someone say "this can be done", or "this is like that was here elsewhere already" without really explaining how or what even. For all I know, such people are not even talking on the same line as I am thinking about it.

This is what I personally understand about it: In order to work around the lack of sandboxing, it is possible to break the normal workflow and "un-scene" some of the content to basically make the game load bare assets dynamically at runtime. Some content also works easily without scenes, such as textures and music. However, as I understand it, for most content such as monsters, items, players, maps, mechanics such as puzzles, other entities such as vehicles or dynamic products of craft, NPCs, markets, traders, transmutators, nested and mutable dynamic environments, external out-of-game API calls, etc. the "un-scening" is only possible if you write an entirely new abstraction layer for this content from scratch and put that in a centralized single scene that handles all the content and "de-objectifies" everything. This abstraction layer will naturally be limited to the very very few things you managed to abstract, and it will in most cases be very laborious to write it even if its scope is very small.

For example, a community contributor might want to create a swarm of ravens as a monster that can cluster together to attack you with swarm intelligence and they can steal random items from your inventory if they hit you and they can also pick up stuff from the environment and drop it on you and several ravens can magically merge to create another monster while they create custom particle effects and change the color of the sky and the player's stats change also. If you write the abstraction layer as a game creator, none of this can be reasonably foreseen and mostly it cannot even be reasonably accounted for even if you foresaw it. The abstraction layer would pretty much look like this: "A monster always is a single hitbox with a single model and a texture and an animation that makes certain sounds and that always moves towards you in a pre-defined manner that cannot be changed and nothing about this can change unless the abstraction layer changes and it can do nothing additionally to that."

This is the problem: such an abstraction layer is basically an engine within an engine. Either it is extremely simplistic and limited, while still introducing additional complexity and requiring substantial efforts to implement. Or else writing it simply takes so long it defeats the purpose. This is especially sad, since the engine offers so many possibilities. And restricting people in the fashion of whitelisting to be able to only make use of a small fraction of it is very very bad.

In the case of loading maps, I don't see how an abstraction layer for scene-less maps can realistically be written at all without writing half the engine yourself. Quodot also doesn't work like this, it is just an import tool. I doubt it can ever work like it. Also I cannot even see how I can even load a player model. Player models usually contain one or multiple meshes, one or multiple skeleton structures, multiple animation sequences, multiple textures and material definitions and mappings and other information. Importing and exporting this information is a very delicate process for most formats that often doesn't even work 100% right in the major editing programs/engines and it then needs touch-ups and manual rework to be fitted in. How can you make a game and then also write all this perfectly working bug-tested loader code for assets that the engine usually loads, but even it always fails to do so to some degree?

I have also heard that you maybe can load scenes but you can whitelist scripts, such that community contributors can only make use of scripts that shipped with the game. Although that would be extremely powerful and helpful, compared to how I just described the situation, I don't believe that feature was even implemented. And also it doesn't sound very secure to me. To make a very dumb and flawed comparison, you could think of scenes as file systems and scripts as files. If you are able to tamper with the filesystem, you can probably easily tamper with where and how files are called as well, regardless of which files are actually whitelisted. There just seems to be so much surface area to attack. I am just saying, if any serious effort had been put into this for the purpose of loading arbitrary community content, then it would have been rather put straight into sandboxing scripts than this, right?

Not knowing all the answers, I can see here and there how I could do this and that to mitigate some and if reaching very very far maybe even all of those problems. But remember: Even if there are clunky solutions to some of those problems, if I am creating a multiplayer game that thrives on community content then most of the time I would be also just handing down those clunks to my community content creators until the end of time. Ideally community contributors would enjoy all the benefits of using Godot as well when they are creating their contributions. That is really "the juice" of the matter.

So you can see I have a hard time wrapping my head around this and I am trying to describe all the deal-breaking issues that would be produced by any sort of solution I can think of.

Community content is the most important aspect of multiplayer games. It makes games thrive and survive. Of course you don't need such content for single player, small casual, vintage/retro or mini games, game jams, showing ads to kids on mobile, or if you are a monopoly and just want to exploit people. But for the rest of it most of the time, that's what a multiplayer game is and should be all about.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Someone knowledgeable and experienced should write an officially sanctioned guide on what is and isn't possible in Godot 4.0 and maybe also 3 to make community content possible in games. This will help game developers to go forward and they don't have do ditch community content out of desperation.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

The guide should take the following scenario as baseline for comparison:

What's the difference between having sandboxing of scripts available and just being able to load any kind of scene into the game from a community server without any security risk, versus however you can load community content as it is now?

The guide should focus on the following:

honestly explain what is and isn't possible for one without reaching, stretching the odds, and extensive additional work
honestly explain what is and isn't possible for the other if you are reaching, stretching the odds, and do extensive additional work
describe all the major limitations and problems that such solutions create

What is and is not?

If something is currently possible in the engine, but this cannot be trivially transferred to apply to community content then it is to be considered either not or not really possible
if something needs to be implemented from scratch either way for content that shipped with the game or that came from the community, then it shouldn't be explained how to implement it, but only how a community content implementation would possibly differ and be limited in possibilities

Use cases:

simple: basically replicate what is already possible in sandboxed old games: Quake 3, Minecraft, Second Life?, Team Fortress, MMORPGs?, Xonotic. Community contributors should at least be able to independently create and distribute maps that are Quake or "90s style simple" and everything that is in them such as sounds, music, textures/materials, models, geometry and abstract entities.
advanced: modern game mechanics and contents should be addressed (Fallout 4, Fortnite, GTA 5, New World), such as crafting systems, physics, items, player stats, abilities and modifiers, in-game level building mechanics (e.g. gathering wood, nails, sheets and so forth to craft shelters), games within games.
very advanced: future use cases for modern games should be addressed, such as external API calls (e.g. to cryptocurrency or neural networks), web sockets, rendering websites, loading videos, doing phone calls, interfacing with communication services, and what not

What is community content?

Community content is content that can be created by players and is distributed by players (usually through community servers, i.e. servers that anyone can create) within and by your game without you doing or changing anything in your game.

maps
player models
UI skins
monsters
items
music and sound
scripting of game logic, like you can use lua in Minetest within the game
NPCs
games within games, e.g. puzzle games to open doors in maps
2D UIs, as used for interacting with door panels or NPCs or custom settings
surface materials, meshes, textures and so forth needed for maps or other things
dynamically loaded external data (videos, images, text)

Please feel free to expand this, it is only the few things that I could think of.

Also I didn't really think of 2D games. But for 2D I think the situation is so simple that it doesn't really raise any such huge problems as with 3D games.

godotengine / godot-docs