Add simulation exercise for CameraCalibration

[thomasfermi]

I think that instead of a video, we could prepare a Carla simulation script for an exercise. First step for the students would still be the test jupyter notebook where calibration is performed on one or more sample images. Vanishing point will be plotted etc.

Then students who have Carla can run a simulation where the car can be controlled manually with the keyboard. But there is also one key that turns on the automated driving. For the lane detector, the initial guess for pitch and yaw can be 5° and 0° but the actual values would be something like 4° and 1.5°. Students shall first run this simulation without camera calibration. They shall see in the simulation how this wrong extrinsic calibration will affect the lane keeping- Next they run the simulation with their calibration. In the simulation, some code will run that executes calibration when the car is not turning (small yaw rate) and moving forwards (large enough speed). the "precomputed grid"(which depends on the extrinsic calibration) of the lane detector shall be changed, based on the new extrinsic calibration and the students shall see how this increases the lane keeping performance. The manual driving is needed since the students will need to drive straight and fast for the camera calibration to run. This exercise is mainly for the experience. All (or nearly) all code of the Carla simulation script shall be supplied to the students.

[MankaranSingh]

@thomasfermi I think the video serves all purposes except the manual driving part. I have added hints in the chapter to dertermine straight lanes with curvature which would be used in the video since there is a curve at one point, so calibration shouldn't run in this part.

Running calra + lane detector togeather makes the gameplay laggy and makes it difficult to control the car manually. But its a good idea to somehow add a script to test lane tracking peformance with wrong extrinsic matrix.

[thomasfermi]

@MankaranSingh Good point regarding the laggy gameplay.

Another idea to show bad performance with wrong intrinsic matrix: We (not the students) could run a simulation with wrong calibration. We store a video and a plot (cross track error over time, or maybe better the trajectory within the lane). Then we create a video and a plot with the calibration that was estimated using the vanishing point method. In the book chapter, at the beginning we show the plot which displays the bad behavior with incorrect calibration. This should also serve to motivate the student to continue reading. We can optionally also embed a video or just paste a link to a video showing the bad performance. At the end of the chapter when the student understands the method, we can show how the lane keeping improves with the calibration.

[thomasfermi]

Closing this, because of the laggy gameplay argument. The other idea that came up here is tracked in #18

[thomasfermi]

Reopened: Instead of manual driving one can use the control code using the Carla map instead of the lane detector, while the CalibratedLaneDetector is still calibrating.

I added code for this here and it worked well for me (Using CalibratedLaneDetector instead of LaneDetector improved performance).

Will test and document this a bit more in the future. Something annoying currently: cg.precompute_grid() takes so much time that the Carla Sim freezes from time to time.... EDIT: Rewrote camera_geometry.py using numba. Get some speed up using that. Will commit soon.

[thomasfermi]

Remaining work to close this issue:

• commit something that speeds up camera_geometry.py (e.g. the numba solution mentioned in the previous post)
• add text to the book that mentions this exercise