Crash due to DepthMapPass and alpha threshold

[ghost]

Hi!

Line 73 in DepthMapPass.as reads: _numUsedStreams = 2;

But it should actually read: (UPDATE 1: Actually, it shoud be 2 and possibly more [skinning data? applies to the other case as well], but in cases where the alpha threshold is used, the UV stream must be set and used in the shader, so the following is just an incomplete temporary fix!) _numUsedStreams = 1;

This breaks depth rendering for anything that has skinned animation. The issue it that the wrong number of streams causes a vertex stream (3, in my case) to be set, even though it is not used in the bound vertex skinning program. There's also no reason (that I am aware of) for the number to be 2, as the use of a texture or the threshold do not require a vertex stream (see code for context)!

UPDATE 1: The only reason to include another vertex stream would of course be to have UVs, but I don't see them being used in the shader, so that is probably the real issue here.

There are possibly other places / combinations with the same behaviour, which I haven't tested. This has been a problem from (at least close to) the release of the first preview version of Away3D 4 and remains in the code base until this day (see release and master branches). I just managed to trace the crash I got to this issue and after setting the number to 1, things finally work!

Best regards.

UPDATE 2: If you enable this quick fix, the skinning for any meshes with alpha threshold will be broken, as I suspected above. If you do not apply it and do not set "Debug.active = true", it will work in the example, but it can cause problems in other places. (In addition to not being able to debug a program, that is.)

[rob-bateman]

this issue should be dealt with in our next milestone update. do you have a simple example that demonstrates the error from your own tests?

[ghost]

Sorry, no, I don't (UPDATE: see my comment below this one). At this point I'd have to modify one of the examples, but you should be able to reproduce it by enabling the alpha threshold of a skinned mesh and then use the DoF 3D filter for the view.

When the corresponding draw call is made, Flash Player should complain about vertex stream 3 to be set and unused. It is set just before the draw call, in the call to setRenderState() (or so).

[ghost]

I've updated the original post with more information about the cause of the issue, but I didn't confirm it yet, so it should be regarded as speculation.

A way to reproduce this issue in Intermediate_CharacterAnimation.as:

• Add "Debug.active = true;" at the top of init(). (The error will be silent otherwise, which is bad, as it can cause problems in other places!)

• Add the following statements to the end of the AssetType.MESH case in onAssetComplete():

        bearMaterial.alphaThreshold = 0.5; // <- Triggers error in DepthMapPass
  
        const filter:DepthOfFieldFilter3D = new DepthOfFieldFilter3D();
        filter.focusDistance = 512.0;
  
        view.filters3d = [filter];

The error should be thrown in drawTriangles() during depth rendering.