Training with CARLA images

[hsarpaydemir]

I am trying to train the model with images from CARLA but the extrinsic matrix that I get from CARLA is used to map from camera coordinates to world coordinates, while the "imageview2ego" method uses vehicle oriented coordinates from what I understood. So calculating the homography matrix between the virtual camera and the actual camera is not working well. Do you have any insights as to how to proceed? Thank you.

[qinjian623]

You need to remap the world coordinate system to the vehicle.

If you have the transformation from vehicle to the world coordinates, along with the position of the lane lines in the world coordinates, you can actually calculate the position of the lane lines relative to the vehicle.

[hsarpaydemir]

Thank you for your response, I actually solved the issue as you've said. One more thing to look out for is to not use the extrinsic matrix provided by CARLA (rgb_cam.get_transform().get_matrix()) as this matrix is used to transform between the world coordinate system and the Camera coordinate system. Instead, one needs to calculate the extrinsic matrix for the vehicle coordinate system with position and rotation relative to the vehicle as opposed to the global coordinate system.