Calculating depth values for FXTOF1 in near mode

mathklk commented 1 year ago

I'm reading depth frames from my AD-FXTOF1-EBZ with the first_frame.py example. I have noticed that - when the camera runs in near mode - the integer values in the depth frame do not actually represent the distance to the object. Or at least the distance is not given in millimeters as I assumed. They're off by some factor larger than 2 (the actual distance is ~700 mm, the measured integer value is ~300) In medium mode however, the integer values do seem to correspond to the distance in millimeters.

I've looked through the docs, but I found no mention on how to calculate the distance from the raw depth frame. Is there some fixed factor to correct the depth frame?

The aditof-demo program shows the distance at the crosshair. That one works in both near and medium mode, so I assume there's some way to calculate the absolute distance even in near mode. I'm currently studying the source code of the aditof-demo, but so far I haven't found the part where the conversion is done.

Thanks for taking your time to read this, any help is appreciated.

rbudai98 commented 1 year ago

Hi! Here is how aditof-demo is calculating the distance, might be helpful: https://github.com/analogdevicesinc/aditof_sdk/blob/4351028dd919525b0c5dccb2b7d1cf44360ca558/examples/aditof-demo/aditofdemoview.cpp#L1160

mathklk commented 1 year ago

@rbudai98 Thanks for your reply! I've stumbled across that line before while looking through the demo.

From what I understand this line continually updates the current depth at the center of the crosshair. The "current depth" is calculated by taking a 70/30 ratio of the previous value and the new raw depth - I assume for a smoother transition.

However, this line does not seem to convert between the raw values and millimeters. The _displayDepthImage method does not differentiate between medium and near mode, so it must happen elsewhere.

I'm trying to understand the demo's source code, but the lack of comments doesn't make it easier ^^ Am I overlooking something obvious here? I assumed the conversion to be an important part of the SDK, so I'm a bit confused it's not clearly documented.

rbudai98 commented 1 year ago

You are right, this line does not differentiate between near and medium mode. That is done on sensor level, in calibration. You can find it her:

And if you check further in the same file you can find the assigned gain value for different modes. Hope it help, Robert

dNechita commented 1 year ago

Hi, Each element (or pixel if you want) in the depth frame represents the distance in millimeters. There is no extra calculation step required, regardless of the mode (near or medium). Can you share how you are measuring the distance to an object?

mathklk commented 1 year ago

Hi @dNechita,

I'm using the latest sd card image with a Raspberry Pi 4 (SDK version v3.1.0). Then I'm running the first_frame.py example which is already on the sd card, see the output below. (I have added one line of code which outputs the depth value of the pixel in the very middle)

pi@aditof:~/workspace/github/aditof_sdk/bindings/python/examples/first_frame $ python3 first_frame.py 
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0517 11:22:35.872848  6518 system_impl.cpp:116] SDK version: 3.1.0 | branch: HEAD | commit: 6a6ab0a
I0517 11:22:35.873095  6518 system_impl.cpp:121] SDK built with websockets version:3.2.3
I0517 11:22:35.873308  6518 sensor_enumerator_raspberrypi.cpp:49] Looking for sensors on the target: Raspberry PI
system.getCameraList() Status.Ok
system.getAvailableModes() Status.Ok
['near', 'medium']
system.getAvailableFrameTypes() Status.Ok
['depth_ir', 'depth', 'ir']
I0517 11:22:35.875828  6518 camera_fxtof1.cpp:133] Initializing camera
I0517 11:22:35.876011  6518 addi9036_sensor.cpp:149] Opening device
I0517 11:22:35.876171  6518 addi9036_sensor.cpp:167] Looking for the following cards:
I0517 11:22:35.876291  6518 addi9036_sensor.cpp:169] unicam
I0517 11:22:36.859403  6518 camera_fxtof1.cpp:189] Camera ID: A1VD99140100
I0517 11:22:36.863641  6518 camera_fxtof1.cpp:197] Camera initialized
camera1.initialize() Status.Ok
camera1.getDetails() Status.Ok
camera1 details: id: A1VD99140100 connection: ConnectionType.OnTarget
I0517 11:22:36.916736  6518 addi9036_sensor.cpp:266] Starting device 0
camera1.setFrameType() Status.Ok
depth_ir
I0517 11:22:36.938117  6518 camera_fxtof1.cpp:222] Chosen mode: near
I0517 11:22:36.938275  6518 camera_fxtof1.cpp:235] Camera range for mode: near is: 250 mm and 800 mm
I0517 11:22:36.938534  6518 camera_fxtof1.cpp:246] Found firmware for mode: near
I0517 11:22:36.938601  6518 camera_fxtof1.cpp:249] Firmware size: 14404 bytes
I0517 11:22:39.430969  6518 calibration_fxtof1.cpp:296] Camera intrinsic parameters:
    fx: 371.147
    fy: 370.963
    cx: 320.207
    cy: 232.821
camera1.setMode() Status.Ok
camera1.requestFrame() Status.Ok
frame.getDetails() Status.Ok
frame details: width: 640 height: 480 type: depth_ir
[[226 227 226 ... 225 225 225]
 [227 226 227 ... 223 223 218]
 [225 226 226 ... 223 215 215]
 ...
 [800 800 800 ... 251 251 253]
 [800 800 800 ... 248 249 249]
 [800 800 800 ... 245 244 248]]
pixel at (320,240) : 205
I0517 11:22:39.627768  6518 addi9036_sensor.cpp:308] Stopping device

During this run, the camera was pointed at a flat sheet of cardboard circa 470 mm in front of the camera. I understand that the 800 values are where the near mode clips or can not measure properly - that is not the problem. The ~200 values are what bothers me.

The aditof-demo works fine, see the screenshot below (crosshair says 460 mm)

grafik

mathklk commented 1 year ago

Ok, found the solution. Everything works fine, there just seems to be a problem with the first handful of frames that arrive. If one ignores the first three frames or so, all following frames have the correct depth in millimeters.

I believe that a certain popular stereoscopic camera has a smilar issue with the first few frames.

dNechita commented 1 year ago

Glad to hear.

analogdevicesinc / aditof_sdk

Calculating depth values for FXTOF1 in near mode #810