vp2RenderDelegate crashes with uniform displayOpacity

[jufrantz]

Describe the bug

A crash may occur when rendering in vp2 an usd mesh defining displayOpacity primvar with uniform interpolation. There is an invalid array access here :

https://github.com/Autodesk/maya-usd/blob/d57109d2689b52cafd00f0cb932405c1e841a204/lib/mayaUsd/render/vp2RenderDelegate/mesh.cpp#L1564

Steps to reproduce

1. Make sure vp2 is in "Smooth Shaded" mode
2. Import attached usd scene as a mayaUsdProxyShape

Specs (if applicable):

• Linux CentOS 7.4
• Maya 2022
• Maya USD 0.9.0
• Pixar USD 21.05

[santosg87]

@jufrantz i have tried a few different things on my side and wasn't able to reproduce this at all. are you able to confirm if this is still a problem for you on the latest releases of the plugin?

[jufrantz]

Hello @santosg87,

Thank you for your answer and sorry for the late reply.

I checked with maya-usd v0.11.0 / USD v21.08 / maya v2022, and the invalid array access is still here, uniform opacity is still misinterpreted.

But there are problems with the scene I attached in the issue. It is too simple to show the problem as a crash, it will most probably only generate an undefined value for transparent flag. Segmentation faults occurred in our case with much bigger scenes that I cannot share unfortunately.

I changed the uniformOpacity.zip so that the loop on trianglesFaceVertexIndices does not break too early (before invalid access). By adding manual bound check in lib/mayaUsd/render/vp2RenderDelegate/mesh.cpp, the problem will show up as a crash (instead of an undefined behaviour), this way for example :

@@ -1724,6 +1724,12 @@
                         int x = _meshSharedData->_renderingToSceneFaceVtxIds[triangle[0]];
                         int y = _meshSharedData->_renderingToSceneFaceVtxIds[triangle[1]];
                         int z = _meshSharedData->_renderingToSceneFaceVtxIds[triangle[2]];
+
+                        if (x >= alphaArray.size() || y >= alphaArray.size() || z >= alphaArray.size())
+                        {
+                            throw(std::out_of_range("out_of_range when reading alphaArray"));
+                        }
+
                         if (alphaArray[x] < 0.999f || alphaArray[y] < 0.999f
                             || alphaArray[z] < 0.999f) {
                             renderItemData._transparent = true;

On our side, we fixed it by changing renderItemData._transparent computation with this more naive code that ignores interpolation and value indirection.

                 if (alphaInterp == HdInterpolationConstant) {
                     renderItemData._transparent = (alphaArray[0] < 0.999f);
                 } else {
-                    for (const auto& triangle : trianglesFaceVertexIndices) {
-
-                        int x = _meshSharedData->_renderingToSceneFaceVtxIds[triangle[0]];
-                        int y = _meshSharedData->_renderingToSceneFaceVtxIds[triangle[1]];
-                        int z = _meshSharedData->_renderingToSceneFaceVtxIds[triangle[2]];
-                        if (alphaArray[x] < 0.999f || alphaArray[y] < 0.999f
-                            || alphaArray[z] < 0.999f) {
-                            renderItemData._transparent = true;
-                            break;
-                        }
-                    }
+                    renderItemData._transparent = (
+                        std::find_if(alphaArray.begin(), alphaArray.end(),
+                            [](float a) {return a < 0.999f;}) != alphaArray.end()
+                    );
                 }
             }

[santosg87]

HI @jufrantz I am still having trouble reproducing this. I am investigating a bit more on the scene.

[jufrantz]

Hello @santosg87, I just saw https://github.com/Autodesk/maya-usd/issues/1687. It is showing the same internal bug but its repro steps should be easier to execute.

[santosg87]

@jufrantz thank you! I will link them and get some more input. :)

[williamkrick]

@jufrantz I believe this is fixed by https://github.com/Autodesk/maya-usd/pull/1906, please re-open if you still have issues.