Sprite3D does not handle model with transparency

[cchuicchui]

• axmol version: Downloaded in 2022-09-29
• devices test on: iOS Simulator
• developing environments
  • NDK version: NA
  • Xcode version: 13.4.1
  • VS/msvctools version: NA
  • cmake version: NA Steps to Reproduce:

1. Modify Physics3DKinematicDemo::init to add the code after line 310:

   floor->setSyncFlag(Physics3DComponent::PhysicsSyncFlag::NONE);

New code:

// static object sync is not needed
floor->setSyncFlag(Physics3DComponent::PhysicsSyncFlag::NONE);

// NEW CODE STARTS BELOW
{
    Physics3DRigidBodyDes rbDes;

    float scale = 12.0f;
    std::vector<Vec3> trianglesList = Bundle3D::getTrianglesList("MeshRendererTest/tree1.obj");
    for (auto& it : trianglesList) {
        it *= scale;
    }

    rbDes.mass = 0.0f;
    rbDes.shape = Physics3DShape::createMesh(&trianglesList[0], (int)trianglesList.size() / 3);
    auto rigidBody = Physics3DRigidBody::create(&rbDes);
    auto component = Physics3DComponent::create(rigidBody);
    auto sprite = Sprite3D::create("MeshRendererTest/tree1.obj");
    sprite->addComponent(component);
    sprite->setPosition3D(Vec3(20.0f, 0.0f, 0.0f));
    sprite->setScale(scale);
    sprite->setCameraMask((unsigned short)CameraFlag::USER1);
    this->addChild(sprite);
}
// NEW CODE ENDS HERE

// create Kinematics
for (unsigned int i = 0; i < 3; ++i)

2. Add the files into tests/cpp-tests/Resources/MeshRendererTest:

tree-model.zip

3. Run test on Physical3D at test case 3:

4. I tested the same changes in 3.17.2 and transparency is shown properly:

NB: Seems that a lot of things crashes since porting to Apple Metal.

Please help to fix.

[aismann]

4. I tested the same changes in 3.17.2 and transparency is shown properly:

Works with Cocos2d-x-4.0?

Seems that a lot of things crashes since porting to Apple Metal.

Only iOS simulator bug? Have you tested it on real iOS devices too?

Plz change title like this: "Sprite3D does not handle 'obj' models with transparency (iOS (?simulator/device?) metal)

[cchuicchui]

It fails with Cocos2d-x-4.0 as well.

My current setup does not support installing in real iOS devices. Is there any setup of cpp-tests such that I can both compile and run in iOS simulator and iOS real devices in one project?

[cchuicchui]

I tried to convert tree1.obj to tree1.fbx (using blender) then convert to tree1.c3t and replaced the obj file to c3t file but nothing is displayed at all.

So the problem is not just relevant to obj file only.

[halx99]

does it works on windows? I have no Mac on my hand to check this issue.

[cchuicchui]

Sorry I don't have Windows on hand...

Please see whether you can reproduce this in Windows... I hope this is not platform specific.

But my overall experience of using 3D model in iOS simulator / iPhone is not very good at all...

[aismann]

Same on windows.

[aismann]

@cchuicchui The 'tree-model.zip' content is free? I want add it to the resource part of test-cpp

[cchuicchui]

@cchuicchui The 'tree-model.zip' content is free? I want add it to the resource part of test-cpp

Yes, it is free

[halx99]

I can confirm, it caused by incorrect blendState set, I have enableBlend for Mesh, then the render result seems correct:

[cchuicchui]

I can confirm, it caused by incorrect blendState set, I have enableBlend for Mesh, then the render result seems correct:

What is the code you used? Can you post here?

[halx99]

The code is:

void Mesh::bindMeshCommand()
{
    if (_material && _meshIndexData)
    {
        _material->getStateBlock().setCullFace(true);
        _material->getStateBlock().setDepthTest(true);
        _material->getStateBlock().setBlend(true);
    }
}

[aismann]

works better now: but not realy transparent (see red circle)

[aismann]

Maybe the title of this issue) is also little bit wrong

[cchuicchui]

Many of the leaves are missing. When I view the model in blender:

[halx99]

Yes, should be other some things incorrect.

[aismann]

Many of the leaves are missing. When I view the model in blender:

@cchuicchui Can you create a c3t from the tree? And test it too

[cchuicchui]

Same result:

The c3t generated is attached here: tree-model2.zip

[cchuicchui]

Would the problem be due to the overlapping of faces constituting the leaves?

[DelinWorks]

Have you tried calling sprite->getMaterial(0)->setTransparent(true) on it? because the transparency functionality has been moved to materials themselves

[aismann]

sprite->getMaterial(0)->setTransparent(true)

@DelinWorks the code gets crashing

[DelinWorks]

I'll see what i can do on my end

[halx99]

works better now: but not realy transparent (see red circle)

I have checked cocos2d-x-3.17.2 same result:

[solan-solan]

The transparent objects (polygons) should be sorted base on distance to the camera, since they renders with depth write function disabled. But it is impossible to do when you draw tree leaves which are intertwined with each other. It is better to apply 'alpha test' in the fragment shader in this case:

if (pixel.a < 0.5)
    discard;

[cchuicchui]

I think twice. Even the incorrect rendering result should not result in less leaves if your drawing of transparency pixels is additive. The leaves in the back should not be over-drawn by background

[cchuicchui]

And to solve the overlap polygon, we may break the polygons into smaller polygons based on their intersections. This will eliminate all overlapping polygons and you may not need to change the shader at all

[solan-solan]

The leaves in the back should not be over-drawn by background

You are right. May be depth write function is enabled for some reason, and foreground polygons was rendered firstly. It could be if render command was not placed to opaque transparent render queue, as example

And to solve the overlap polygon, we may break the polygons into smaller polygons based on their intersections

Do you suggest to break polygons on background and foreground along the intersection lines, then draw them in the sorted order? To my understand it is not simple task for the engine. I doubt that even Unity has this feature from scratch

[aismann]

Only an idea: Is the tree correct shown on Unity?

[halx99]

import the model to unity scene, and the results:

[solan-solan]

Looks like common blending with depth write disabled.

And I do not see that there is polygon sorting exists. Fix me if I wrong, but farther polygon is rendered after nearer polygon in the red square on the picture:

[DelinWorks]

@solan-solan is correct, you can't sort the vertices if they are transparent, because they are in a static model obj they cannot be sorted from front to back unless you create a transparent pass to the renderer which is not the best option, it would would make sense if you were to render something like glass where it's not entirely transparent... the correct solution is to instead write a fragment shader that discards full transparent pixels i.e.

if (pixel.a == 0.0)
    discard;

that's the solution used in minecraft's code, used for blocks like glass blocks, it's not viable nor efficient to create a render pass just for glass, the solution they resided on is discarding 0 alpha pixels which is both viable and efficient for the situation at hand. and displays really will for blocks in front and behind.

[cchuicchui]

Is something missing here?

[cchuicchui]

Originally I was trying to look for a low-poly tree that is "real" enough. After some searching, this tree model is good looking yet with minimum number of faces.

If my purpose is just to find some good looking trees and put into my world, is there any other tree models that work in Axmol?

This looks really nice to me: https://github.com/cocos/cocos-example-cartoon-vegetation

Any chance we can reproduce this in Axmol?

[solan-solan]

Is something missing here?

I believe that you can render good-looking tree if you apply custom fragment shader with proposed technicians (it would be better to get one of Axmol fragment shaders and add upper lines with discard. cpp_test will show how to create custom material).

But I really do not understand why depth write is not disabled while common transparent with blending renders. It looks bug anyway

[solan-solan]

I have got the following:

with this code:

auto sprite = Sprite3D::create("res/test/tree1.obj");
sprite->setPosition3D(Vec3(0.0f, 0.0f, 0.0f));
static_cast<Sprite3D*>(sprite->getChildren().at(1))->getMaterial(0)->setTransparent(true);
static_cast<Sprite3D*>(sprite->getChildren().at(1))->getMaterial(0)->getStateBlock().setCullFaceSide(CullFaceSide::NONE);
sprite->setScale(10.f);
sprite->setCameraMask((unsigned short)CameraFlag::USER1| (unsigned short)CameraFlag::USER2| (unsigned short)CameraFlag::USER3);
this->addChild(sprite);

And you need to add the following change in the Axmol engine:

But leaf polygons are not sorted, you need use above aproach to avoid this

[aismann]

Looks good with this code (physics is working also)

CCMeshRenderer.cpp is the old one (without: material->setTransparent(oldmaterial->isTransparent()); )

        auto sprite = Sprite3D::create(tree1);
        sprite->addComponent(component);
        sprite->setPosition3D(Vec3(20.0f, 0.0f, 0.0f));
        sprite->setScale(scale);
        this->addChild(sprite);

        static_cast<Sprite3D*>(sprite->getChildren().at(1))->getMaterial(0)->setTransparent(true);
        static_cast<Sprite3D*>(sprite->getChildren().at(1))->getMaterial(0)->getStateBlock().setCullFaceSide(CullFaceSide::NONE);
        sprite->setCameraMask((unsigned short)CameraFlag::USER1 | (unsigned short)CameraFlag::USER2 | (unsigned short)CameraFlag::USER3);

[aismann]

Not only a bug...also a wrong cpp-tests code issue? @halx99 should we add the labels testcode and engine ?

[aismann]

@cchuicchui , @solan-solan @DelinWorks @halx99 , can we close it?

@cchuicchui
Maybe a second obj (same kind, more stress) for stable tests code will be nice?

[solan-solan]

@aismann CCMeshRenderer.cpp is the old one (without: material->setTransparent(oldmaterial->isTransparent()); )

This thing is needed if you use light in the scene. The shader will be recreated in the code

I have light in my scene and transparent is not copied to the new material without that line

[aismann]

@solan-solan I add whatever is needed to close this issue ;) Good work (thanks all)!

[aismann]

@cchuicchui , @solan-solan @DelinWorks @halx99 , Have a look on it => test it and than: Please give for closing a "Yes close it" ;)

[halx99]

lgtm

[solan-solan]

I think it is ready now!

[cchuicchui]

Should we set transparency as default?

What is the overhead if we set this to default? Or is there any disadvantage if we set this to default?

[aismann]

Should we set transparency as default?

What is the overhead if we set this to default? Or is there any disadvantage if we set this to default?

@cchuicchui It's more a (new) discussion not an issue. Plz vote ;)

[cchuicchui]

Yes, this issue is resolved. Thanks all for your hard work!