search

google-coral / edgetpu

Coral issue tracker (and legacy Edge TPU API source)
https://coral.ai
Apache License 2.0
422 stars 124 forks source link

Raspberry Pi HQ camera and Coral Dev Board Mini #535

Open asmirnou opened 2 years ago

asmirnou commented 2 years ago

Description

I'd like to connect Raspberry Pi HQ camera and Coral Dev Board Mini through MIPI CSI-2 camera interface. Their buses and voltages are compatible except for the connectors are different: 15-pin and 24-pin accordingly. I made an adaptor linking two boards together, making sure all lines match another side.

The camera sensor is controlled through I2C interface by the kernel driver. I took one from the Raspberry Pi repository and modified it for the Mendel OS. The driver works in tandem with Mediatek MIPI-CSI2 receiver driver.

The results of my work can be found in the following GitHub gist.

What I manage to achieve is that imx477 sensor is detected, Mediatek MIPI-CSI2 receiver driver recognises the sensor and creates v4l2 node. /dev/video1 video device is present and can be controlled via v4l2-ctl utility, switching the formats (RG12/RG10) and modes. Both sensor and receiver start to stream, I can see that MIPI signal passes through the data lanes.

However the receiver hardware doesn't capture the data. The last logs messages are:

[  119.154942] videobuf2_common: __vb2_wait_for_done_vb: call_qop(000000009f11aaf0, wait_prepare)
[  119.163628] videobuf2_common: __vb2_wait_for_done_vb: will sleep waiting for buffers

Then the program waits for something to hit the buffer, but nothing happens. Upon termination, the output file is empty.

Apparently, the receiver hardware does not correctly parse the MIPI packets. Either data format is wrong and/or frequency. The configuration is written to the receiver's registers. It was originally set to YUV format, while imx477 transmits data in RAW12 format. I followed the comments left in the receiver's code to change the format, but they are quite meagre. I tried to find the right setting by guess and clearly didn't succeed.

I'm opening this issue so that others can contribute to my efforts. If someone knows more about the Mediatek MT8167 SoC and can provide documentation on the MIPI CSI receiver registers or if someone can spot a bug in my code, I would greatly appreciate it. After all, the Coral boards should support more than one camera :)

Click to expand! ### Issue Type Feature Request ### Operating System Mendel Linux ### Coral Device Dev Board Mini ### Other Devices _No response_ ### Programming Language _No response_ ### Relevant Log Output _No response_
hjonnala commented 2 years ago

Hello @asmirnou

I appreciate your efforts here and thanks for sharing your work!

This would require supporting the full ISP pipeline along with MIPI CSI configuration change to support the RAW12 format. Unfortunately, our team is prioritized on other work but we will pass to our dev team for consideration.

All the Best!

joshfox10 commented 1 month ago

@asmirnou have you had any luck with this?