search

duckietown / gym-duckietown

Self-driving car simulator for the Duckietown universe
http://duckietown.org
Other
45 stars 16 forks source link

Extrinsic camera calibration #173

Open liampaull opened 5 years ago

liampaull commented 5 years ago

It was noted at AI-DO that there is a huge discrepancy in the extrinsic calibration of the camera in the simulator vs. the real robot.

For a side by side comparison, look where the horizon is in this simulated video vs. this video from a robot in the robotarium

This huge bias could cause very negative results in sim-to-real transfer I fear.

bhairavmehta95 commented 5 years ago

Yes agreed, we really need to discuss this in person though, because it's a much bigger bug than it looks - I spent a few weeks on this before AIDO and it was a nightmare to get even this far.

liampaull commented 5 years ago

yikes

liampaull commented 4 years ago

@bhairavmehta95 can we have a meeting about this?

liampaull commented 4 years ago

actually my best guess is that it's not (just) the extrinsic calibration but that the field of view is wrong for sure 

CAMERA_FOV_Y = 42

since it's pulled from the camera specs without the fisheye lens

liampaull commented 4 years ago

I also noticed that the values in distortion.py are different than the ones burned to the vehicle by default

bhairavmehta95 commented 4 years ago

These are new parameters; they were taken from this file much earlier this year (and much earlier than April, which is when these were pushed).

liampaull commented 4 years ago

got it - ideally they should be pulled from the same place...

pmirallesr commented 3 years ago

I just want to raise awareness about the FOV issue. In our own sim2real experiment we have resorted to putting papers under the camera support piece to change its inclination so that it more closely mimicks that of the simulator. This simple trick did a great difference in our sim2real efforts

liampaull commented 3 years ago

We have a process now of calibrating the camera in the simulator