webaverse / app

Web metaverse client
https://app.webaverse.com
MIT License
338 stars 211 forks source link

Improving light and shading in Webaverse. #3636

Open GuiltyRegicide opened 1 year ago

GuiltyRegicide commented 1 year ago

Currently. Webaverse does not have a source for reflections. Because of this, any characters or assets created using the standard shader will have Black reflections. This causes strange discoloration with certain models.

Alongside missing reflections, Webaverse still does not have some key modern lighting features that would drastically improve the visual quality.

Some ways to implement this and improve the look of Webaverse would be:

Reflection probes: no direct implantation within Three.js but still possible to code in. something similar to unity’s Reflection Probes. These would Ideally be actively updating based on the location of the character in the scene.

Light probes: https://threejs.org/docs/#api/en/lights/LightProbe

Optional HDRI environment maps: https://threejs.org/examples/webgl_materials_envmaps_hdr.html This affects everything in the scene and uses the HDRI for additional lighting. The only issue is that with Webaverse the effect can be too strong. And does not affect the toon shader.

Skybox Cubemaps: Skybox textures can do double duty and act as the reflection map.

A nice to have additional feature would be to have all of these be editable from a creators perspective.

An example of a character who's values were tweaked to be metallic and reflective. image

avaer commented 1 year ago

This is a great feature to add, but I think strong multi-scene + rendersettings swap support is a prerequisite before we start on it.

I could be wrong, but I suspect any such env maps will cause a shader pipeline recompile in THREE.js, so enabling/disabling this dynamically will cause hitching. Therefore adding this feature will likely degrade/delay multiplayer when crossing multiple scenes, or possibly make Upstreet unplayable.

I don't know of any UGC platform combines the fine tweakability of render settings with seamless world blend like we do (not even VRChat does this, it's a very long loading screen as everything gets recompiled, and you cannot communicate across worlds).

So before implementing this we should first come up with a compilation plan for how to dynamically swap these maps in multiplayer without any hitching.

Alternatively we could simply forbid such render settings changes in multiplayer, but that would defeat much of the point of this feature. That would still work in isolate parcel mode though.

Either way, this should be thought through with a clear plan + research results before starting.