Why the car can't appear in the camera area?

[LiuXuan66]

The prompt is ‘add a black car on the front on the scene‘. I use my own data. I don't know where the problem is？It seems that my camera's posture is incorrect, it cannot face forward on the x-axis.The mcNerf training is right. Is there a fixed code parameter written during the camera generation process？

[LiuXuan66]

16 Perhaps I misunderstood the role of trans_mat。

[yifanlu0227]

13 Maybe the same problem? You will have to align your world coordinate's origin with the ground.

[LiuXuan66]

13 Maybe the same problem? You will have to align your world coordinate's origin with the ground.

Does this mean that the ground is based on the vehicle coordinate system of the first frame? If that's the case, then I have already made the association because my ground feature coordinates are based on the vehicle coordinate system and do not need to be converted like Waymo.

[yifanlu0227]

Could you check it?

[LiuXuan66]

Could you check it?

Yes, my own dataset is similar to Waymo, and its car coordinate system is also set at the bottom of the vehicle, with the coordinates facing the same direction as Waymo

[LiuXuan66]

You can see that the vehicle is generated in the correct world coordinate system, but the coordinates of the camera's blender are obviously incorrect, and its y-axis (triangular tip) is not pointing upwards.

[yifanlu0227]

I will give you a code snippet to visualize your poses (pose_bounds.npy and cam_meta.npy). I think you should check them first

[yifanlu0227]

https://github.com/yifanlu0227/ChatSim/blob/main/chatsim/foreground/Blender/utils/blender_utils/utils/visualize_pose.py

[yifanlu0227]

You need to run it on your PC with a display

[LiuXuan66]

Thanks a lot. I'll try it.

[LiuXuan66]

You need to run it on your PC with a display

I have visualized the pose and I am not sure if the world coordinate system in the interface is FLU, so is this pose correct?

[yifanlu0227]

There is a segment indicating the up direction of your camera, so you can see how your camera is positioned (if you give the correct camera convention)

FLU is a camera convention, not a world coordinate system property. You should know your camera convention in your data format. But at least for now, it's not a normal visualization result. If your dataset is the same as waymo's, with xy plane determining the ground and +z pointing to the ground facing upwards, then probably your camera convention is incorrectly specified, so the cameras are facing crooked, but their trajectory looks reasonable.

[LiuXuan66]

Thank you very much for your answer. I will carefully examine the issue again

[LiuXuan66]

16 Finally, I obtained the correct pose by annotating trans_mat, and the vehicle can appear within the camera range.