search

3dtof / voxelsdk

VoxelSDK - an SDK supporting TI's 3D Time of Flight cameras
BSD 3-Clause "New" or "Revised" License
107 stars 71 forks source link

Question: Raw_Processed_Frame data parsing; Spatial median filter parameters #76

Closed SankethR closed 8 years ago

SankethR commented 8 years ago

Hello,

I used the "CameraSystemSaveStreamTest" to save an XYZI Point cloud Frame file consisting of 100 frames and then provided this as an input file argument to the "CameraSystemReadStreamTest", but the program exits because it is unable to get necessary frame generators to read stream.

camerasystemreadstreamtest_error

How could I parse the data stream generated by "CameraSystemSaveStreamTest" to metric data & intensity data? The current output of the file is a binary stream. I read a few earlier posts on data parsing, but I am unable to figure this out by myself.

Are there any other examples available for this? Also, how would the exported data change if "RAW_FRAME_PROCESSED" is chosen?

SankethR commented 8 years ago

@gadiyar I have an application where the camera is on a movable platform, so to prevent Motion artifacts, I want to use a spatial median filter. However, the parameters are not clear to me.

I was able to setup the filter in the "DepthCapture.cpp" example as shown in the link here: https://github.com/3dtof/voxelsdk/wiki/Voxel::DepthCamera. My questions are:

1) What do the parameters - "stability", "deadband", "deadbandStep", "halfkernelSize" of the median filter do? A neighborhood of 3x3 or 5x5 has to be set for a median filter to work. How does the halfkernelSize, ranging from values [1-100] correlate to a Pixel neighborhood?

2) If I want to do some additional pixel validation scheme, say use the Signal-to-Noise Ration (SNR) as shown here: iq_graph Is there such a possibility in the VoxelSDK to create such a filter profile? If not, the Amplitude/Ambient value for each pixel could be a useful pointer to the "confidence of measurement", isn't it?

3) How can I access the data for conversion to depth in [m] & Amplitude [grayscale value] from a RAW_PROCESSED_FRAME, as exported by "DepthCapture.cpp". The sample from a single Frame capture looks like this: raw_frame_capturedata I saw this in the OPT8320 SBAS748 document: bytegrouping

So according to my understanding, the data parse is happening this way: raw_frame_capturedata_parse Is this correct?

I would appreciate your feedback in this regard, I am stuck on these fundamental aspects since last week. Thanks.

SankethR commented 8 years ago

@gadiyar An Addition to question (1) of the above post, because the camera is always looking at a changing scene, could you give some ideas on keeping the deadband value adjustment bounded? Due to the deadbandstep adjustment, it could occur that the deadband value could very easily increase in an unbounded way.

gadiyar commented 8 years ago

For your second question: Amplitude is the same as Confidence - just a change in terminology.

FRAME_RAW_FRAME_PROCESSED has the frame data parsed into phase/amplitude/flags. FRAME_DEPTH_FRAME has the phase data translated into radial distances assuming the modulation frequency is correctly provided.

You can either use the phase data and manually multiply with the phase-to-distance scaling factor, or use the depth data directly.

Your understanding of the depth format is correct. However this is useful only if you're attempting to decode the FRAME_RAW_UNPROCESSED_FRAME data manually. If you start with FRAME_RAW_PROCESSED_FRAME, you can directly access the phase and amplitude values for each pixel.

I will get back to you shortly for your issue with the CameraSystemReadStreamTest and for the Median Filter question.

gadiyar commented 8 years ago

For the issue with CameraSystemSaveStreamTest, the log seems to indicate that no frames were saved. The file saved by CameraSystemSaveStreamTest is the same as a .VXL and can be opened with VoxelViewer.

SankethR commented 8 years ago

Hello Anand, a clarification to what you have mentioned:

Your understanding of the depth format is correct. However this is useful only if you're attempting to decode the FRAME_RAW_UNPROCESSED_FRAME data manually. If you start with FRAME_RAW_PROCESSED_FRAME, you can directly access the phase and amplitude values for each pixel.

I parsed the raw unprocessed frame manually, but I noticed that instead of [C0A0 ... C7A7], [P0F0 ... P7F7], [C8A8 ... C15A15], the camera output is instead [C0A0 P0F0 ... C3A3 P3F3], [C4A4 P4F4 ... C7A7 P7F7]

OP_MODE = 0 (DVP) OP_DATA_ARRANGE_MODE = 1 (Continuous)

Is this correct? If I change OP_DATA_ARRANGE_MODE to 0, the output would be [C0A0 ... C7A7], [P0F0 ... P7F7], [C8A8 ... C15A15]?

gadiyar commented 8 years ago

That's correct - OP_DATA_ARRANGE_MODE = 1 is for continuous data (C0A0 P0F0... format) while OP_DATA_ARRANGE_MODE=0 is for the group-by-8 format.

SankethR commented 8 years ago

Perfect.

For your second question: Amplitude is the same as Confidence - just a change in terminology.

Are there some ball park figures that you have for these settings? Maybe from an existing test report? AMPLITUDE_THRESHOLD and SATURATION_THRESHOLD

pixel_confidence

gadiyar commented 8 years ago

Amplitude threshold is up to you - you can set it based on the reported amplitude you see.

You wouldn't normally need to set iq_scale or iq_scale_en.

I need some time to get back to you with a value for saturation threshold.

SankethR commented 8 years ago

Okay, thanks @gadiyar. I am also awaiting your response on: 1) Saturation threshold 2) What do the parameters - "stability", "deadband", "deadbandStep", "halfkernelSize" of the median filter do? A neighborhood of 3x3 or 5x5 has to be set for a median filter to work. How does the halfkernelSize, ranging from values [1-100] correlate to a Pixel neighborhood? An Addition to question (2), because the camera is always looking at a changing scene, could you give some ideas on keeping the deadband value adjustment bounded? Due to the deadbandstep adjustment, it could happen that the deadband value could very easily increase in an unbounded way.

larrylisky commented 8 years ago

I don't recommend messing with stability deadband and deadband steps. Those are simply hysteresis keep the image stable but if object is moving they will distort. They will likely be removed. Half kernel is (kernel_size -1)/2. Typically 3x3 works fine.

SankethR commented 8 years ago

@larrylisky Thanks for the explanation.

SankethR commented 8 years ago

@gadiyar Assuming without setting anything to the SATURATION_THRESHOLD variable (Default: 0), what happens to the phase/distance data when the pixel is saturated? Does it give 0 m or does it show the an arbitrary maximum value based on the UNAMBIGUOUS_RANGE setting? How do I recognise error pixels in the depth map?

SankethR commented 8 years ago

@gadiyar @larrylisky What do the 4 bits of ambient data signify? Is it as shown in the table below? In my sample analysis, I get a value of 13, so that would mean the pixels are saturated as per the table below. Is my understanding correct??

ambient_flag

gadiyar commented 8 years ago

@SankethR The 4 bits of ambient give a single number indicator of the non-modulating component of voltage on the pixels. This is the sum of the ambient light, pixel offsets, and the non-demodulated component of ToF illumination. The output value decreases with increase in voltage. A value near 0 indicates saturation and a value near 15 does not.

For saturated pixels, the phase is 0xFFF and the amplitude is 0. This happens even without setting a value for saturation threshold. (I think the SDK has a bug where it is not masking these saturated pixels and subtracts out the per-pixel offsets for these - this will be fixed separately).