dji-sdk / Mobile-SDK-Android

DJI Mobile SDK for Android: http://developer.dji.com/mobile-sdk/
Other
978 stars 580 forks source link

It is not possible to calculate precise FoV having only 35 mm equivalent focal length #1216

Open ComBatVision opened 1 year ago

ComBatVision commented 1 year ago

How to calculate precise vertical and horizontal field of view of camera in photo and video mode in SDK 4, if it returns only 35mm equivalent focal length, but no CMOS size or its original aspect ratio or at least its maximal resolution to calculate aspect ratio?

To be able to calculate FoV we need one of this options: 1) Real matrix size in mm to calculate crop factor and then real focal length and then arctan for angles from real width or height and real focal length. 2) Original aspect ratio to calculate equivalent 35 mm matrix size with the same diagonal and then use its dimension in original aspect ration and 35 mm focal length. 3) Maximal resolution to calculate original aspect ratio and do the same way as in p.2

But DJI SDK has nothing except 35 mm equivalent focal length and currently selected aspect ratio which may not be equal to original.

Could you add CMOS size or native aspect ratio or photo resolutions list into SDK please.

DJI-William commented 1 year ago

You can check the CMOS size on the spec section on our official website. E.g. For Mini 2. image

ComBatVision commented 1 year ago

Do you propose to hardcode all payload sensor sizes manually in the code and update application each time new drone or camera appears?

Should I match it by drone name? Where to get all list of drone names to have no issue in mapping?

What about 60fps video mode in Mavic 2, Mavic Air 2 and Mavic Mini 2 when matrix become cropped about 1.5x times? In this case matrix size from spec does not help.

I think sensor size in each photo and video mode should be returned by SDK, otherwise it is impossible to make vertical and horizontal FoV calculations correctly and precisely. What is the reason why it is not in SDK?

ComBatVision commented 1 year ago

By the way, I found a hidden CameraKey PHOTO_ASPECT_RATIO_RANGE, which returns all supported aspect ratios and allows to build FoV calculation logic without hard-coding sensor size, based on assumption that if result of this key contains 3:2 aspect ratio, then original matrix aspect ratio is 3:2 otherwise it is 4:3.

Then I calculate FoV based on equivalent width or height taken from aspect ratio and diagonal 43.27mm, cropping it to selected aspect ratio PHOTO_ASPECT_RATIO from SDK in photo mode and wide video mode or 2160 pixels in narrow video mode.

But the only thing which does not work is 60fps mode in some drones, which crops effective sensor size about 1.5x times in 60fps mode and this can not be handled by any attribute in SDK. The only option which I found is to force 30fps mode always on drone connection to my application, but this is bad practice for user.

How to handle 60 fps matrix cropping and how to determine which drones do this cropping and which are not?

DJI-William commented 1 year ago

PHOTO_ASPECT_RATIO only returns you a ratio when capturing photos. It seems to lack of information for videos.

ComBatVision commented 1 year ago

Yes. We even can calculate FoV for video, but not fpr 60fps.

Video in Wide mode works the same way as photo, bu cropping full matrix to 16:9.

Video in Narrow mode uses 3840x2160 pixels out of 4056x3040 or 5472×3648. We can just crop 16:9 part of 4k in the center of matrix.

But 60fps mode is unpredictable. In some drones it works as usual narrow video, but on other drones it crops matix more to be able to upscale 2.7k to 4k 60 fps because of drone CPU limitations.

We need a straight forward way in the SDK to detetmine active matrix size in all modes or calculated vertical and horizontal FoV.

DJI-William commented 1 year ago

Sorry, this cannot be done because there is no such protocols.

ComBatVision commented 1 year ago

@DJI-William is it possible to add some attribute into drone firmware which will contain crop factor for 60fps? If it is possible, then CMOS size will be also useful instead of hardcoding size by drone name.

DJI-William commented 1 year ago

I am pretty sure it cannot be added.

ComBatVision commented 1 year ago

Ok, then we will keep workaround to force 30fps video framerate on our application startup.