search

maiermic / robot-cameraman

Robot cameraman similar to Soloshot or Pixio
MIT License
13 stars 5 forks source link

acceleration and (maximum) speed of rotation should depend on distance of tracked object to the camera and the focal length #13

Open maiermic opened 2 years ago

maiermic commented 2 years ago

If the tracked object is closer to the camera, the camera has to rotate faster to keep up with the movement of the tracked object. If the tracked object is further away, the camera has to rotate slower. Otherwise, it may over-rotate (rotate faster than the tracked object) and loose the tracked object.

TrackingStrategy, AlignTrackingStrategy, and SearchTargetStrategy may should adjust the (maximum) speed and acceleration to the distance of the object to the camera and the current focal length of the camera.

The distance of the tracked object to the camera can be estimated by the bounding box (i.e. detection candidate in the live view image) and the current focal length of the camera: https://photo.stackexchange.com/questions/12434/how-do-i-calculate-the-distance-of-an-object-in-a-photo/12437#12437

The current focal length can be received/deduced from the headers of the live view (see zoomRatio of ExHeader1 or ExHeader2).

formula

# see https://photo.stackexchange.com/a/12437/89169
distance_to_object_mm = \
    (focal_length_mm * real_object_height_mm * image_height_px) \
    / (object_height_px * sensor_height_mm)

# convert px to mm
mm = \
  (distance_to_object_mm * px * sensor_height) \
  / (focal_length * image_height_px)

# calculate rotation angle
a = distance_of_object_to_image_center (from px to mm)
b = distance_of_camera_to_object_mm
α = math.asin(a/math.sqrt(a*a + b*b)) * 180 / math.pi

visualization

Note: The figure above shows the camera (top view) and the live image of the camera. The camera should be moved in such a way that the destination is in the center of the live image. In the shown example, the destination has to move to the right and down (red arrows) in the live image. By panning the camera to the left, the destination moves right. By tilting the camera up, the destination moves down. The camera has to be panned by α degrees and tilted by β degrees (not shown in the figure) to get the destination to the center of the live image. p is the horizontal (left/right) distance of the object to the center of the live view image. If the camera is panned α degrees, p should get (approximately) zero. The same applies to the vertical (up/down) distance t and the tilt-angle β, i.e. if the camera is tilted β degrees, t should get (approximately) zero. The camera points to the (center of the) object if p and t get zero.

maiermic commented 2 years ago

I got wrong results in the calculation, since I used the equivalent focal length instead of the actual focal length of my Panasonic Lumix DMC-LF1:

Focal Length (actual): 6 - 42.8mm
Focal Length (35mm equivalent): 28 - 200mm
maiermic commented 2 years ago

The calculation/estimation of the distance of camera to object and the calculatation of the angle to point camera to object has been added in commit d0496c5e157eefea5e3ba406e6bc559ceec36ba8. However, it is not used yet to change the acceleration or (maximum) speed of rotation.

Besides, the zoom ratio alone may be used to estimate the angle (with sufficient accuracy) based on the maximum angle (object near the edge of the image) when the smallest focal length is used, see https://github.com/maiermic/robot-cameraman/blob/d0496c5e157eefea5e3ba406e6bc559ceec36ba8/robot_cameraman/distance.py#L131