search

raspberrypi / rpicam-apps

BSD 2-Clause "Simplified" License
420 stars 228 forks source link

High resolution is not supported by IMX477 #477

Closed RagdollCar closed 1 year ago

RagdollCar commented 1 year ago

Trying to record a video with IMX477, 4056x3040, 30fps. According to the Sony product information this resolution should be supported, but it isn't. https://www.sony-semicon.com/files/62/pdf/p-13_IMX477-AACK_Flyer.pdf

Is it possible to record videos in a such resolution? The highest resolution that worked for me was 2028x1080. I have read in some resources that resolutions higher than FullHD might be not supported by RPI4, but I'd like to ask about the reason of not supporting it. Is it a problem with the drivers, cpu or any other?

root@ragdoll:/home/ragdoll/repo/videocamera/build# libcamera-vid --width 4056 --height 3040 --framerate 30
Overriding H.264 level 4.2
Made DRM preview window
[1:35:13.419530901] [2242]  WARN IPAManager ipa_manager.cpp:114 Public key not valid
[1:35:13.422038657] [2242]  INFO Camera camera_manager.cpp:299 libcamera v0.0.0+4171-9b860a66
[1:35:13.496307996] [2246]  INFO RPI raspberrypi.cpp:1425 Registered camera /base/soc/i2c0mux/i2c@1/imx477@1a to Unicam device /dev/media3 and ISP device /dev/media1
Mode selection:
    SRGGB10_CSI2P 1332x990 - Score: 12552.2
    SRGGB12_CSI2P 2028x1080 - Score: 10179.8
    SRGGB12_CSI2P 2028x1520 - Score: 9096
    SRGGB12_CSI2P 4056x3040 - Score: 42000
[1:35:13.521862651] [2242]  INFO Camera camera.cpp:1028 configuring streams: (0) 4056x3040-YUV420 (1) 2028x1520-SBGGR12_CSI2P
[1:35:13.522800502] [2246]  INFO RPI raspberrypi.cpp:805 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 2028x1520-SBGGR12_1X12 - Selected unicam format: 2028x1520-pBCC
ERROR: *** failed to start output streaming ***
6by9 commented 1 year ago

The bare IMX477 sensor can achieve 4K2K @ 60 frame/s, but you'll note that the also state

CSI-2 serial data output (MIPI 2lane/4lane, Max. 2.1 Gbps/lane, D-PHY spec. ver. 1.2 compliant

The Pi HQ camera module connects up 2 of the 4 CSI-2 data lanes as that is what the normal Pi variants expose (Compute Modules expose all 4 lanes on CAM1). It then runs them at 900Mbit/s/lane as the receiver is D-PHY v1.0 nominally running at max 1Gbps/lane.

As reported by libcamera-hello --list-cameras, this translates to a max of 10fps at 4056x3040.

pi@raspberrypi:~ $ libcamera-hello --list-cameras
Available cameras
-----------------
0 : imx477 [4056x3040] (/base/soc/i2c0mux/i2c@1/imx477@1a)
    Modes: 'SRGGB10_CSI2P' : 1332x990 [120.05 fps - (696, 528)/2664x1980 crop]
           'SRGGB12_CSI2P' : 2028x1080 [50.03 fps - (0, 440)/4056x2160 crop]
                             2028x1520 [40.01 fps - (0, 0)/4056x3040 crop]
                             4056x3040 [10.00 fps - (0, 0)/4056x3040 crop]

The H264 encoder on the Pi is rated at level 4.0, which translates to 1080p30. Level 4.2 for 1080p60 can be achieved under some circumstances. 4056x3040 @ 30fps would translate to 48260 macroblocks per frame, and 1447800 macroblocks per second, so up into level 6. https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels

RagdollCar commented 1 year ago

Thank you for your response. I've just realized that my RPI4 has only CSI and I need CSI-2, so I need to get an extension board and connect it to my rpi4 (or maybe it requires a compute module 4)? For example this one: https://botland.store/raspberry-pi-cm/18213-raspberry-pi-cm4io-extension-board-for-raspberry-pi-cm4-5904422368708.html

I have listed my cameras:

root@ragdoll:/home/ragdoll# libcamera-hello --list-cameras
Available cameras
-----------------
0 : imx477 [4056x3040] (/base/soc/i2c0mux/i2c@1/imx477@1a)
    Modes: 'SRGGB10_CSI2P' : 1332x990 [120.05 fps - (696, 528)/2664x1980 crop]
           'SRGGB12_CSI2P' : 2028x1080 [50.03 fps - (0, 440)/4056x2160 crop]
                             2028x1520 [40.01 fps - (0, 0)/4056x3040 crop]
                             4056x3040 [10.00 fps - (0, 0)/4056x3040 crop]

As I understand my IMX477 should work with 4056x3040, 10fps? I tried this command and it also crashes:

root@ragdoll:/home/ragdoll# libcamera-vid -n --width 4056 --height 3040 --framerate 10
Overriding H.264 level 4.2
[0:12:18.912900171] [1286]  WARN IPAManager ipa_manager.cpp:114 Public key not valid
[0:12:18.915357024] [1286]  INFO Camera camera_manager.cpp:299 libcamera v0.0.0+4171-9b860a66
[0:12:18.995894526] [1290]  INFO RPI raspberrypi.cpp:1425 Registered camera /base/soc/i2c0mux/i2c@1/imx477@1a to Unicam device /dev/media3 and ISP device /dev/media0
Mode selection:
    SRGGB10_CSI2P 1332x990 - Score: 12552.2
    SRGGB12_CSI2P 2028x1080 - Score: 10179.8
    SRGGB12_CSI2P 2028x1520 - Score: 9096
    SRGGB12_CSI2P 4056x3040 - Score: 2000
[0:12:19.021589277] [1286]  INFO Camera camera.cpp:1028 configuring streams: (0) 4056x3040-YUV420 (1) 4056x3040-SBGGR12_CSI2P
[0:12:19.022622114] [1290]  INFO RPI raspberrypi.cpp:805 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 4056x3040-SBGGR12_1X12 - Selected unicam format: 4056x3040-pBCC
ERROR: *** failed to start output streaming ***
6by9 commented 1 year ago

Thank you for your response. I've just realized that my RPI4 has only CSI and I need CSI-2, so I need to get an extension board and connect it to my rpi4 (or maybe it requires a compute module 4)? For example this one: https://botland.store/raspberry-pi-cm/18213-raspberry-pi-cm4io-extension-board-for-raspberry-pi-cm4-5904422368708.html

That is the CM4 IO board. Totally irrelevant for a Pi4B. CSI and CSI-2 are used interchangeably. There was a CSI-1, but it's long since defunct. Any current sensor will have a CSI-2 interface.

As I understand my IMX477 should work with 4056x3040, 10fps? I tried this command and it also crashes:

As per my comment about the H264 encoder, it is limited at level 4.2. Look it up in https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels and you'll see it says a maximum of 8704 maximum macroblocks (16x16 for H264) per frame. Your requested 4056x3040 is 254 x 190 maccroblocks, or 48260. 48260 > 8704, therefore the encoder will refuse to start.

4056x3040 frames can be delivered at 10fps as raw images, but they can not be hardware H264 encoded.

RagdollCar commented 1 year ago

Thanks, I found also this response from the past so now I understand the limits. https://github.com/waveform80/picamera/issues/348