We suggest a 120 degree to 160 degree camera lens for Donkeycar so the car, which is close to the ground, can still see both sides of the track. Such a lens creates a distorted image. This is generally ok in that the neural network can still produce good predictions. However, it has a number of disadvantages in other use cases;
It hampers efforts to create more general model using images from many users because they may have different camera lenses and each camera has different inherent distortions.
It makes it difficult to use an image to figure out the relative position of an obstacle because distances and angles are distorted.
Project:
Implement a utility to walk the user through getting the necessary image samples of a checkerboard and then calculate the camera calibration matrix. Ideally it automatically snaps the images so the user does not have to press a button for each image, then it allows the user to decide if any images should be dropped and it more images should be taken, until they are happy.
Note that this problem has been solved many times so we are not looking for a brand new implementation. There are many open source projects that have code to do this; we want to use that code and integrate it into donkeycar like the donkey createcar or donkey calibration command; so add a new command like donkey camera-calibration and save the user's calibration in their mycar folder.
Once we can save calibrations, write a part that will use the calibration to correct the camera image. It's run() method will take the uncalibrated image and then use the calibration parameters to correct it.
The corrected image should have an bird's eye view; see Issue #910 and jump to the section entitled A Neural Network using 2D Lidar data. That project needs a bird's eye view image so it can merge it with 2D lidar data. The bird's eye view is useful in other ways; it can be used to estimate the angle and distance from an obstacle for instance.
Once we have a part that will produce bird's eye view images, integrate it into the deep learning template so we can create an autopilot using bird's eye view images.
We suggest a 120 degree to 160 degree camera lens for Donkeycar so the car, which is close to the ground, can still see both sides of the track. Such a lens creates a distorted image. This is generally ok in that the neural network can still produce good predictions. However, it has a number of disadvantages in other use cases;
Project:
donkey createcar
ordonkey calibration
command; so add a new command likedonkey camera-calibration
and save the user's calibration in their mycar folder.A Neural Network using 2D Lidar data
. That project needs a bird's eye view image so it can merge it with 2D lidar data. The bird's eye view is useful in other ways; it can be used to estimate the angle and distance from an obstacle for instance.