GLSL Shader support

[sirkitree]

Something I'm working on right now is creating video textures based off of GLSL shaders. I'm using https://magicmusicvisuals.com/ for this, which allows me to capture a given shader's runtime code as video, then import that into Neos and use that as a texture for a mesh. It's a great effect.

However, I'm wondering if this could be more performant if there were a way to run the GLSL shader code in Neos directly and apply it to a mesh.

Attached is a sample tube4.txt

Perhaps this is a duplicate of #63 - if so I wonder if this could be revisited to see if there is a better way to do this now that time has passed.

[BlueCyro]

I second this, and while I know it's rather fruitless to compare neos to something like VRChat because of how different the games are, the one thing that I miss from it is the beautiful shader worlds that can be created, it would be fantastic to see worlds like these in neos.

[shiftyscales]

It is a highly desirable feature for many, but to implement it- it needs to be done right in a way that suits Neos' core design principles.

To quote Frooxius from the initial issue #63:

Unfortunately this is not currently possible, it's one of the major features that's bugging me on Neos currently, as there's no clean solution for this. Unity doesn't expose a way to load own shader code at runtime, so there's no clean way to do that with it as Neos' runtime.

I might make it possible by pre-compiling them within Unity, but that's kinda "dirty" way, that doesn't fit Neos' philosophy and could cause issues in the future.

Alternatively I'll look into another way using native API's or perhaps some support from Unity itself. Could potentially switch to a different runtime environment / engine in the future as well, but that's a bit of a drastic measure, that'd require significant amount of work.

[BlueCyro]

I mean if you're writing shader code, ideally you're going to need to do it outside of neos for the most granular control (typing in VR is pretty bad, not to mention even thinking of making an in-world scripting language similar to logix for shaders). The unity precompile could be a good stop-gap so that we at least have the feature until the middleman can be removed.

[shiftyscales]

Stop-gaps don't work though as we don't ever want to break compatibility with worlds/assets. When incorporating a feature into Neos, it has to be something that at its core won't need to be significantly altered for many years, and if it is altered- extended or replaced by a compatible substitute.

[Frooxius]

Yeah, this is sort of duplicate of the shader support here: https://github.com/Frooxius/NeosPublic/issues/63

I cannot guarantee that the supported language will be GLSL. It's specific to certain graphic runtimes. For example we're currently using DirectX 11 on Windows, which doesn't support GLSL, so we'd have to transpile those shaders to this which can get complicated.

There's still the bigger underlying problem of not being able to support shaders in the first place. Not much has changed there unfortunately, Unity still doesn't have any nice API's for this. Neos' shader handling has been reworked into packages that can solve some of the problems (like variant generation and keeping the source for recompilation), but this puts severe limitations on what we can do in the future.

It's not about typing in VR, but about being able to compile and generate the shaders in the first place. Say we wanted to make a visual builder for shaders, so you can make them in-game like LogiX scripts. Without that ability we can't do that.

Even worse, shaders tend to be specific to a particular rendering pipeline and with Neos we have to consider long term compatibility for that. With such limitations, it's hard to devise a solution that wouldn't lead to us having to break many (or even all) user-made shaders at some point.

Anyway I'm going to close this issue, since there's already the topic on being able to support shaders that's more general, rather than a specific language: https://github.com/Frooxius/NeosPublic/issues/63

Feel free to continue discussion there, but unfortunately this is one of the "up in the air" kind of features. We're going to have to wait and see if new options open up as we work on other things.