(Custom Shaders) Way to Import Shaders

[MR-Alex42]

Is it possible to import your own shaders in Neos?

[Frooxius]

Unfortunately this is not currently possible, it's one of the major features that's bugging me on Neos currently, as there's no clean solution for this. Unity doesn't expose a way to load own shader code at runtime, so there's no clean way to do that with it as Neos' runtime.

I might make it possible by pre-compiling them within Unity, but that's kinda "dirty" way, that doesn't fit Neos' philosophy and could cause issues in the future.

Alternatively I'll look into another way using native API's or perhaps some support from Unity itself. Could potentially switch to a different runtime environment / engine in the future as well, but that's a bit of a drastic measure, that'd require significant amount of work.

[ScribbleFen]

Is there anything in particular tying you down to Unity, or is it just the generally daunting task of switching to another engine? For extensibility and customizability, I think y’all should probably consider https://github.com/godotengine/godot since it already has a lot of things exposed like adding new shaders, and for anything else you might need to add, the engine is free and open-source so you can just code it in or download other contributor’s modifications. Idk what kind of work y’all would need to transition over, but I’d be happy to help in any way if y’all ever consider it; this is a really cool project ^^

[Firepal]

@ScribbleFen Dynamic shader compilation in Godot is sluggish and blocks rendering until complete. This is a massive no-no for VR, until a solution is found Godot is not viable

[ScribbleFen]

@Firepal Oh okay sorry guess I missed that >.< but I did a quick search and it looks like at least when Godot 4 comes out, with Vulkan, they should be able to do caching and threading so the shader compilation doesn't hold up rendering or anything, along with a whole slew of other helpful features. But I guess until then yeah, it sounds like Godot isn't very viable I guess...

[Firepal]

@ScribbleFen I hope that's right. Custom shaders are of high value to me as well and I hope a good solution for supporting them in NEOS can be come-to. I just want real-time pretty pictures ;w;

[ScribbleFen]

@Firepal Yessss! I mean my main thing personally is I'm just really into procedural textures--I hate having to bake out and uv map traditional textures when I could just write a shader to do it more easily and with higher fidelity (anything with gradients or sharp shapes especially, for example). Here's hoping!

[Frooxius]

Switching to another rendering engine is a pretty huge task, that will take months of work, so it's not something we'll do lightly. We have considered Godot, but it doesn't quite suit our needs, so we don't have any plans switching to that one at the time.

Doing custom shaders properly in context of Neos will very likely require a custom system which we can control, to ensure not only good performance, but also security and long term content compatibility.

We don't have anything specific to announce yet, but we're continually looking and working towards some potential solutions, but it is a pretty long term endeavor.

[gentlecolts]

If I may: I don't know much about the engine's current pipeline but if it were somehow possible to combine existing shaders, that could add a lot of flexibility without having to implement custom shaders. For example: what if users could use the output of an unlit material to drive a texture channel on a PBS material, such as emission. In this example that'd allow taking advantage of the unlit shader's masking to potentially create interesting emissive effects. I can definitely imagine other creative ways this could be used if it were possible

[shadowpanther]

@gentlecolts see https://wiki.neos.com/OfficeHours:Geenz:2021-07-08 for the planned way to make more customizable shaders.

[Frooxius]

@gentlecolts Unfortunately that's not possible with how current shaders work, you'd need to create a new shader in order to do that. Current shaders do not output a texture, they are responsible for actually drawing the object on the screen directly.

That's going to be implicitly handled by the ability to make custom shaders, where you can combine the functionality you need.

[VegaDeftwing]

In #3453 it was said

our custom shader system will be graph based like logix once we have it

I'd just like to put in my 2¢ by saying while I think having a graph system for shaders will be preferable for the majority of users, it may also either be overly limiting to the users who want it most or result in many shaders being made with poor performance when users are knowledgeable enough to optimize them otherwise. For example using if (and for to a lesser extent) is much simplier in a node based system than forcing users into the pseudo-functional style of using step, mod, andfrac of more efficient shaders.

I know above it was was said:

I might make it possible by pre-compiling them within Unity, but that's kinda "dirty" way

I think a good middle ground might be making it possible and easy enough to make and import custom nodes. To be transparent, I think my biggest concern is that traditionally heavy things - like feedback via render textures - may not be possible while this is exactly what allows for much of the complexity seen in VRChat avatars.

Similarly, I could foresee not letting users write/complile the shaders directly causing problems/limits on what's possible as attributes of the shader and how it interacts with the engine would be inaccessible, most notably all of the shaderlab commands

Of course, take literally everything I'm saying with a full cup of salt: I'm a VRChat user that's heavily into the shader community over there, looking at joining Neos for the same reason as everyone else at the moment.

[Geenz]

Our custom node-graph based system will not be limiting beyond some incredibly specific criteria. We intend to expose the full shader pipeline to users, with the geometry pipeline being the sole exception - namely in the sprit of ensuring future upgradability and long term compatibility with our pipeline as we move towards Forward+ and mesh shaders.

If I were to set out the specific stages we want:

• Mesh (Vertex/Geometry/Hull/Domain)
• Fragment
• Compute

For Mesh we're looking at emulating on incompatible platforms either by compiling out the specific stages into their own stages, or emulating via compute.

I think the biggest boon here is tooling - and if we can tokenize existing Unity shaders for this for easier import, we will absolutely look into it.

[Geenz]

For compute, you will be able to submit buffers to a compute shader, and get buffers out that you can use for whatever you like.