Fail to import ActiveLightSensor in SAPIEN 3.0

[chenjy2003]

System:

• OS version: Ubuntu 20.04
• Python version (if applicable): Python 3.9.18
• SAPIEN version (pip freeze | grep sapien): sapien @ https://github.com/haosulab/SAPIEN/releases/download/nightly/sapien-3.0.0.dev20231122-cp39-cp39-manylinux2014_x86_64.whl#sha256=9a7d6fe666341ddae8c5a5d9f4449245e61eed5a643fdd24ee9ec462a449faf0
• Environment: Server with xvfb

Describe the bug Fail to import ActiveLightSensor in SAPIEN 3.0. The error is "ImportError: cannot import name 'Renderer' from 'sapien.core'".

To Reproduce Steps to reproduce the behavior (use pastebin for code):

1. from sapien.sensor.activelight import ActiveLightSensor

Expected behavior To import normally.

Screenshots

Additional context Add any other context about the problem here.

[fbxiang]

I believe the ActiveLightSensor is deprecated and may be removed completely in SAPIEN 3 release. We will only support the GPU accelerated sensor. An example using this sensor is https://github.com/haosulab/SAPIEN/blob/dev/manualtest/stereodepth.py

[chenjy2003]

Thanks for your reply! I tried this example. The result image and point cloud look pretty good!

However, I encountered a problem when I tried to load novel objects to the simulator, other than those simple build-in geometries like boxes and spheres.

For example, I want to replace the box in the example code with another box of side length 0.05m, which is stored in an .obj file model.zip. But in the stereo image, the box got missed accidentally. Therefore, I want to consult with you what is the correct way to import new objects. My code is shown as follows:

box_mesh = trimesh.load("data/assets/simple_geometry/box_0.05/model.obj")
render_component = sapien.render.RenderCudaMeshComponent(box_mesh.vertices.shape[0], box_mesh.faces.shape[0])
render_component.set_vertex_count(box_mesh.vertices.shape[0])
render_component.set_triangle_count(box_mesh.faces.shape[0])
render_component.set_triangles(box_mesh.faces)
render_component.set_material(sapien.render.RenderMaterial(
   base_color=[0.8, 0.2, 0.2, 1.0],
    # specular=0.8,
    roughness=0.01,
    metallic=1.0,
))
# render_component.set_physx_body_type("kinematic")
entity = sapien.Entity()
entity.add_component(render_component)
entity.set_name("box_0.05")
entity.set_pose(sapien.Pose(p=[0.05, 0.26797, 0.09], q=[1, 0, 0, 0]))
scene.add_entity(entity)

Thank you in advance for your assistance!

[fbxiang]

The CudaMesh should only be used for deformable objects. If you are building a body that does not need to be deformed, follow https://sapien-sim.github.io/docs/tutorial/basics/create_actors.html

[chenjy2003]

Thanks for your reply! Unfortunately, our project has to contain deformable objects for tactile simulation. Does the current StereoDepthSensor support deformable objects? If so, could you please elaborate on how to correctly import deformable entities so that they can be rendered correctly? Thanks a lot!

Additionally, I encountered some issue when loading objects that does not need to be deformed. In this tutorial, the only way to add an object from a mesh file is by actor builders, which seems incompatible with abd_component. If I build an object by an actor builder, and then add the abd_component, I will encounter an error assert self.entity is None or self.entity.scene is None. I'm wondering if there are some other methods to add objects from mesh files without using actor builders.

[fbxiang]

abd_component seems to be from an internal extension library which is not released yet. You should contact the library developer directly. Anything that can be rendered with the ray tracer should work with StereoDepthSensor. The deformable body rendering is under development. If you have a specific use case you want me to look at, please provide a minimal reproducible code sample with SAPIEN code and create a new issue.

[chenjy2003]

Thanks for your reply!

The above error is because the actor_builder method must set scene.

However, it seems that the render_component method needn't to set scene. So I'm wondering if this render_component method in the tutorial can support loading objects from mesh files.

# create render body for visualization
render_component = sapien.render.RenderBodyComponent()
render_component.attach(
    # add a box visual shape with given size and rendering material
    sapien.render.RenderShapeBox(
        half_size, sapien.render.RenderMaterial(base_color=[*color[:3], 1])
    )
)
entity.add_component(render_component)