Regression: some URP shaders have too many variants in 2021.2

[hybridherbst]

Describe the bug Not sure if this is a regression in URP or in glTFast, I believe the former is more likely. When adding the glTFast shaders to the list of "Always included" shaders, building to Android on 2021.2.0f1 fails with the error message: Shader Graphs/glTF-metallic-Premultiply-double has too many Shader variants(201326592).To resolve this, go to Project Settings > Graphics and remove this Shader from the Always Included Shaders list.For more explicit control over how Unity includes specific Shader variants in a build, use a ShaderVariantCollection.

As the goal for the glTFast shaders would indeed be to be always included (so that all models can be loaded), I think this is a blocker for using glTFast for runtime loading on 2021.2 right now.

To Reproduce Steps to reproduce the behavior:

1. make a project on 2021.2 with URP targeting Android
2. add the glTFast shadergraph shaders to the list of always included shaders
3. try to build

Expected behavior Building works, actually error message prevents building

[atteneder]

Thanks for reporting...that's a real bummer :/

Let's brainstorm solutions:

• Splitting up shader by certain shader keyword (GAMMA would be a good candidate, since you never need both linear and gamma workflow variants)
• Convert certain shader keywords into properties (downside is additional complexity in final shader code and thus a chance of performance impact)

On top of that an Editor tools/script that auto-generates a sane sub-set of variants based on your required material features. Even if a project needs to deal with all features, certain combinations are unlikely ever used (e.g. emissive, occluded and transmissive at the same time).

This artice might be useful. Imho the problem is less glTF(ast) specific, but more generic (interesting read about this is shader permutations part 1 and part 2)

Let me know your thoughts.

[hybridherbst]

I actually think it needs to be fixed in Core, as it seems that shader stripping is applied after checking for Always Included shaders. Reported to Unity as (Case 1378545) [Shaders][URP][Regression] Build is prevented with shaders in Always Included list

[hybridherbst]

For reference,

2020.3

2021.2

[atteneder]

Note: While debugging other material problems (transparency/blending in HDRP >=10) I noticed that some material features that used to require a separate shader graph can now be controlled via shader keywords. That might be helpful for this issue as well.

[hybridherbst]

Yes, that and also since ShaderGraph supports Built-In starting with 2021.2 the added complexity of those shaders also goes away. Shaders still have to be individually tuned per RenderPipeline though (e.g. the material options for the URP target are entirely different to the material options for the HDRP target, even if they have the same names).

[atteneder]

@hybridherbst Looking at it!

Strangely I couldn't reproduce it (glTFast 4.4.9, Unity 2021.2.7f1 with URP 12.1.2). I now ticked off "strip unused variants" in URP's global settings and now I have to wait a while :)

Here's the Unity issue you reported

edit: second build with stripping disabled also worked (took over 20 minutes)

[atteneder]

@hybridherbst Is this still a problem? glTFast 4.5.0 uses less shader graph versions. We should re-evaluate this issue.

[ROBYER1]

I was relieved to remove a lot of variants when using URP 12 with gltFast 4.5.0+, keeping an eye on build times here to see if there is a big improvement

[hybridherbst]

I think the underlying issue still isn't fixed in Unity though (it's still in the "Known Issues" list of Really Bad Things In Releases)

It might just be under the one million variants it needs to be to pass the broken check now, I'll also try that out

@ROBYER1 the issue isn't that that all these many variants where actually built, it's that Unity did first check how many could potentially be there, and abort if that was too many, instead of counting and checking after stripping. (still of course build times might go down a bit with full glTFast shader builds with less variants!)

[ROBYER1]

I noticed the same issue with Unity also. The only benefit I had was in Unity 2021.2 with GLTFast 4.5.* and URP 12 I didn't have to use so many legacy variants.

Keeping an eye on this as we got the aborting during builds if I slapped all the variants in rather than just the ones I use so I got lucky in our app use case.

[hybridherbst]

Thing is, if you want to load generic glTF models at runtime (e.g. you don't know what users will add) you definitely want to include the entire set of glTFast shaders as always included ones - otherwise some models will look wrong. Fingers crossed that Unity actually fixes this soon...

[pzhine]

Unity allows me to build with all of the glTFast shaders included, but it takes a long time (30-40 min on a fast machine). Does anyone know where Unity puts the built shaders, so I can archive those files and just drop them into subsequent builds? I'm building for iOS and Android...

[hybridherbst]

Subsequent builds will reuse the shaders on the same machine, they're getting cached directly into the Library. Only the first build on a machine (or when you changed the shaders) should take this long.

[pzhine]

Actually, if I don't check "Scripts Only Build" they get compiled every time. I'm on 2021.3.1 and glTFast 4.7.0

[hybridherbst]

That would be a bug, please report that to Unity. It doesn't matter if "Scripts Only" is on or not, compiled shaders will be cached unless your disk is full. Shader stripping though will not be cached (but shouldn't take excessively long - if it does, still please report a bug).

[pzhine]

Ok, thanks, I will file a bug. For reference, it is the "Preparing variants for compilation" that takes forever.

[hybridherbst]

Yeah, I believe that's a known issue in 2021.2 with URP since they added a ton of multi_compile debug variants that all need to be stripped away at build time and that's not efficient / not cached currently. Can't hurt to report another bug

[pzhine]

Done! https://fogbugz.unity3d.com/default.asp?1425665_8e8453tjnsmolq17

[pzhine]

Everyone who is struggling with this bug, please vote for the fix here: https://issuetracker.unity3d.com/issues/ios-shaders-are-recompiled-on-subsequent-builds

[atteneder]

Closing, since this was fixed by the URP devs:

It's now fixed in Universal RP 12.1.5 (2021.2.14f1), 13.1.6 (2022.1.0b10), 14.0.2 (2022.2.0a9), and above.