Crash while rendering

[Flova]

The rendering crashes at the getCameraImage with the following error:

terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at

This happens both with the panda3d and pyrender rendering engine.

[Flova]

This seems related to #8 but the mentioned fix in #9 does not change this behavior. The scene also only contains the base floor plane and a humanoid robot urdf model.

[ikalevatykh]

Hi @Flova, could you send the code example

[Flova]

Wow that was fast. I post a link in a minute. Not that the code is still WIP.

[Flova]

The file is located here: https://github.com/bit-bots/wolfgang_robot/blob/feature/camera/wolfgang_pybullet_sim/src/wolfgang_pybullet_sim/simulation.py

The call is in line 270.

Edit: I forgot to push. The line changed due to this.

[ikalevatykh]

Could you provide a minimal example without external dependencies. Just loading a urdf and render.

[Flova]

Here is a stripped-down version with fewer deps and only the essential parts.

#!/usr/bin/env python3
import math
import pybullet as p
import pybullet_data
from pybullet_rendering import RenderingPlugin
from pybullet_rendering.render.pyrender import Renderer
import numpy as np

class Simulation:
    def __init__(self, gui):
        self.gui = gui
        self.paused = False
        self.gravity = True

        self.client_id = p.connect(p.GUI)

        p.setGravity(0, 0, -9.81)

        self.time = 0

        # Loading floor
        p.setAdditionalSearchPath(pybullet_data.getDataPath())  # optionally
        self.plane_index = p.loadURDF('plane.urdf')
        p.changeDynamics(self.plane_index, -1, lateralFriction=1, spinningFriction=-1,
                         rollingFriction=-1, restitution=0.9)

        # Loading robot
        flags = p.URDF_USE_INERTIA_FROM_FILE + p.URDF_USE_SELF_COLLISION + p.URDF_USE_SELF_COLLISION_EXCLUDE_ALL_PARENTS
        self.robot_index = p.loadURDF("../../../wolfgang_description/urdf/robot.urdf", flags=flags)

        self.timestep = 1 / 240

        p.setRealTimeSimulation(0)

        self.camera = Camera(self.robot_index, self.client_id)

    def step(self):
        self.time += self.timestep
        p.stepSimulation()
        self.camera.getImage()

class Camera:
    def __init__(self, robot_index, client_id):
        self.robot_index = robot_index
        self.client_id = client_id
        self.counter = 0

        self.frame_rate = 1

        renderer = Renderer()
        plugin = RenderingPlugin(client_id, renderer)

    def getImage(self):
        com_p, com_o, _, _, _, _ = p.getLinkState(1, 12)
        rot_matrix = p.getMatrixFromQuaternion(com_o)
        rot_matrix = np.array(rot_matrix).reshape(3, 3)
        # Initial vectors
        init_camera_vector = (0, 0, 1) # z-axis
        init_up_vector = (0, -1, 0) # y-axis
        # Rotated vectors
        camera_vector = rot_matrix.dot(init_camera_vector)
        up_vector = rot_matrix.dot(init_up_vector)
        # Compute view matrix
        view_matrix = p.computeViewMatrix(com_p, com_p + 0.1 * camera_vector, up_vector)

        # Convert 3x4 projection matrix to 4x4 projection matrix
        fov, aspect, nearplane, farplane = 80, 1.0, 0.1, 100 # TODO realictic vals
        projection_matrix = p.computeProjectionMatrixFOV(fov, aspect, nearplane, farplane)
        print(p.getCameraImage(
            500,
            500,
            view_matrix,
            projection_matrix),
            500,500)

if __name__ == "__main__":
    s = Simulation(True)

    while True:
        s.step()

Edit: Reduced code overhead

[ikalevatykh]

Hi @Flova, sorry for late response.

You should load the plugin before any modifications in simulation world (loadURDF, etc.).

class Simulation:
    def __init__(self, gui):
        self.gui = gui
        self.paused = False
        self.gravity = True

        self.client_id = p.connect(p.DIRECT)
        self.plugin = RenderingPlugin(self.client_id, PyrRenderer())

        # code below ...