Geometry for videos

[satwikkottur]

Includes the following tasks:

1. Clean up geometry code for lane detection ~ 8 hrs
2. Port the geometry for video, making suitable adjustments ~ 8 hrs
3. Make the geometry for videos work ~ 8 hrs
4. Let tracker take help of geometry ~ 16 hrs
5. Ideas for generating targets ~ (?)
6. Use optical flow information for geometry estimation ~ (?)

[satwikkottur]

• Cleaned up the code required for vanish point and boundary estimation ~ 8 hrs
• Ported the code and make changes for it to function for videos ~ 8-10 hrs

[satwikkottur]

• Cleaned up the code related to estimation of vanish point and boundaries ~ 8 hrs
• Ported code to make it work for videos, making necessary changes ~ 8 hrs
• Crude geometry estimation done for videos ~ 5 hrs

Refine methods using optical flow / background for better estimation

[satwikkottur]

1. There are multiple ways of estimating the geometry of the scene.
   • Using Gabor filters: The approach follows the paper. This method has been cleaned up in the original project and ported for the videos. The results are not really good (except for one or two cameras).
   • Using line segments and vanishing point estimate: It is the classical approach of using line segments and finding the dominant intersection points for vanishing point estimation. This part of the code needs cleaning in the original project as well as porting, testing for the videos. Estimated time: 16 hrs

[satwikkottur]

Automatic estimation of geometry not yielding good results. For proof of concept, manual geometry markings have been used (for camera 48).

1. Setting up the framework for manual marking, visualization (debugging) of geometry ~ 8 hrs
2. While trying to incorporate geometry into particle tracker, lot of issues popped up wrt tracker alone. Handling rogue particles, re-spawning particles going out of bounds, etc. ~ Debugging them took time ~ 15 hrs

[satwikkottur]

The estimation of optical flow through the previous implementation was terrible. Needed a better yet fast technique.

1. Searched for a optical flow library that gives good estimates and runs in reasonable time. Setting and trying the demos ~ 4 hrs
2. Incorporating these various implementations of optical flow into the project ~ 4 hrs
3. Constructing the mean flow and visualizing it, for tracklet bounding boxes ~ 8 hrs
4. Setting up the infra-structure for writing down matches ~ 4 hrs

TO-DO:

1. Come up with methods to estimate geometry from activity (either background / optical flow).
2. Visualize tracklet trajectories and come up with methods to use it for geometry estimation.

[satwikkottur]

1. Saving and visualizing the trajectories for the tracklets ~ 2 hrs
2. Estimating geometry from activity - (just vanishing point using ransac) ~ 5 hrs

TO-DO:

1. Continue with geometry estimation from activity.
2. Constraint tracker to use geometry information for robustness.

[satwikkottur]

@kukuruza @Lotuslisa I am kind of stuck without ideas as to how we can incorporate geometry into tracking, any suggestions ? Right now I am using particle based tracking.

Will probably try other trackers, just to make sure this is the best choice. Thanks in advance.

[kukuruza]

What about creating more particles in the most probable (size-wise) direction? I thought that's what you were going to do..

[satwikkottur]

Hmm right, that's the idea but I am unable to materialize it into code. Let's discuss this tomorrow, if we are meeting and I am still stuck. Thanks.