light sensitivity related to optical flow

[PeaceNira]

slight changes in light can effect tracking leading to jumps in points, not ideal for tracking. currently using the Lucas–Kanade method for predicting movement.

[lakshyeahh]

Proposed Solution:

To improve the tracking stability under varying light conditions, you can implement the following changes:

1. Preprocess the Input Frames:

   • Grayscale Conversion: Reduces color noise and simplifies the image.
   • Histogram Equalization: Ensures a more consistent brightness distribution.
   • Gaussian Blurring (Optional): Reduces high-frequency noise.

   Example Code:

   import cv2
   
   def preprocess_frame(frame):
      gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
      equalized = cv2.equalizeHist(gray)
      return equalized

2. Use Optical Flow with Pyramid Scaling:
   If not already implemented, you can enable pyramid scaling within Lucas–Kanade. This allows tracking across multiple scales, improving robustness in challenging conditions.

   lk_params = dict(winSize=(15, 15),
                   maxLevel=3,  # Use pyramids for better tracking
                   criteria=(cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10, 0.03))

3. Adjust Point Initialization Sensitivity:
   Re-initialize tracking points only if the movement is significant. This reduces false jumps caused by lighting changes.

4. Integrate an Illumination-Invariant Feature Detector:
   You can use advanced detectors (like ORB or SIFT) instead of raw optical flow points to ensure that features remain consistent under light variations.

[PeaceNira]

I've implemented all of these technologies already 😅