Neos-Metaverse / NeosPublic

A public issue/wiki only repository for the NeosVR project
196 stars 9 forks source link

Texture / Asset Streaming #924

Open Enverex opened 4 years ago

Enverex commented 4 years ago

There was a discussion about this the other night in Discord but I thought it's worth opening a GH officially for anyone else curious about it.

Currently, Neos appears to load every asset into RAM when you enter a world. While that's not an issue with smaller worlds, it just annihilates PCs when you have giant worlds. The main problem is that these assets get loaded even if they aren't used, just as long as they exist in the map somewhere. So for example on a map I'm working on at the moment, the world is segmented into 14 unique regions, all but 1 of them will always be disabled. Despite this, everything gets loaded and this results in obscene levels of RAM and VRAM usage.

Lowering the max resolution of all world textures to 1024 brings the map back into the realms of usable, but at the expensive of everything now looking terrible (especially so in VR). Here's the RAM and VRAM usage, going from a max of 1024 res, back to each texture's original resolution (typically 2048 and 4096): image

Without this functionality, we're stuck either with large worlds, but using a few textures across the whole thing, or just small worlds with an equally small amount of textures to match them (or splitting everything up into different worlds entirely, but that goes against the idea of creating one giant immersive environment).

Unity and Unreal's engines both already natively have support for this. VRChat added support for it back at the Unity 2018 upgrade at the start of the year.

References:

Frooxius commented 4 years ago

This could potentially be implemented at some point, but the main question is how to control which mipmaps get loaded at the time. It will likely require some level of instrumentation by the content builder to allow for this.

There are alternatives to this approach, like Virtual Texturing, which is a lot more powerful and general, but that's also a much bigger feature. Another method that can improve things that's native to Neos is Cascading asset dependency tracking. If you have a culling system that disables and enables things as you move around that would naturally load and unload assets: https://github.com/Frooxius/NeosPublic/issues/183

I'm not too sure what other streaming methods you'd want for other non-texture assets, since there's not much to stream generally, as the assets don't have the same concept of mipmaps and distance based detail.

Enverex commented 4 years ago

The "Cascading asset dependency tracking" would definitely solve the immediate issue and leave only edge-cases as problematic (but that at least gets rid of the majority of performance problems in terms of large worlds). The mipmap based texture streaming would then just be a nice performance enhancement on top of that.

"but the main question is how to control which mipmaps get loaded at the time. It will likely require some level of instrumentation by the content builder to allow for this." - Isn't this camera/view based? Much the same as the LOD system works, but rather than being configured per-object, it's configured simply as a single world setting.