search

initBasti / NanoPC-T4_armbian_configuration

My configuration for Linux Kernel development on the FriendlyElec NanoPC-T4 with the latest kernel version.
GNU Affero General Public License v3.0
2 stars 0 forks source link

[Q] Unable to get V4L2_SUBDEV_FORMAT_ACTIVE #1

Closed avafinger closed 3 years ago

avafinger commented 3 years ago

Thank you for your work on the driver.

I am facing the following error with libcamera:

[0:21:28.498784457] [950]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+2127-5988661c
[0:21:28.524573852] [951] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[0:21:28.524697221] [951] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
Using camera platform/ff110000.i2c/i2c-1/1-0010 ov13850

Did you have this error? I have tested with mainline, collabora and a backport of rkisp1 from 5.11 with the same results. Do you have any suggestions or tips I could try?

initBasti commented 3 years ago

Hello,

Yep, I currently have that error as well but I am able to capture frames with it. The missing ioctl, in this case, is VIDIOC_SUBDEV_G_SELECTION. Adding that ioctl to the driver before sending it to the mainline is on my TODO list. The two errors that you get are not hard failures when you look here, you will be able to see that they are only used to get the crop and the bounds of the active pixel array. If they fail the code just continues.

But if you are having trouble recording any video at all, then the actual problem is a different one. Can you post the complete output of the following command: LIBCAMERA_LOG_LEVELS=0 cam --camera=1 --capture=1

And also check if you are able to record a video with the following command: cam --camera=1 --capture=1 --stream height=1080,width=1920,pixelformat=NV12,role=video

If that works then the underlying problem is that the sensor provides a resolution that is bigger than the maximum supported resolution for the image signal processor. And libcamera is currently unable to negotiate that format mismatch properly. I have currently a patch in the making that fixes that issue, look here. I should have mentioned that in the text, that libcamera relies on that patch, I will add that asap.

Greetings, Sebastian

avafinger commented 3 years ago

Hi Sebastian, I am now with my customized kernel with rkisp1 from 5.11 , seems I have broken it. For the main test, I used the collabora kernel 5.10.rc1 and I was able to stream video with mjpg-streamer (yes, dark and green), I will have to rebuild it and test again, but this kernel is not good for the NanoPi M4, the board i have here.

I will follow your progress.

Is it possible for you to push your kernel tree to github?

avafinger commented 3 years ago

Using kernel 5.11.rc2 (did not touch rkisp1):

LIBCAMERA_LOG_LEVELS=0 cam --camera=1 --capture=1

LIBCAMERA_LOG_LEVELS=0 cam --camera=1 --capture=1
[0:13:43.994638487] [882] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyLinux"
[0:13:43.994927524] [882] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyThread"
[0:13:43.996400126] [882] DEBUG IPAModule ipa_module.cpp:330 ipa_rkisp1.so: IPA module /usr/local/lib/aarch64-linux-gnu/libcamera/ipa_rkisp1.so is signed
[0:13:43.996707247] [882] DEBUG IPAManager ipa_manager.cpp:238 Loaded IPA module '/usr/local/lib/aarch64-linux-gnu/libcamera/ipa_rkisp1.so'
[0:13:43.998786215] [882] DEBUG IPAModule ipa_module.cpp:330 ipa_rpi.so: IPA module /usr/local/lib/aarch64-linux-gnu/libcamera/ipa_rpi.so is signed
[0:13:43.999091002] [882] DEBUG IPAManager ipa_manager.cpp:238 Loaded IPA module '/usr/local/lib/aarch64-linux-gnu/libcamera/ipa_rpi.so'
[0:13:43.999519745] [882] DEBUG IPAModule ipa_module.cpp:330 ipa_vimc.so: IPA module /usr/local/lib/aarch64-linux-gnu/libcamera/ipa_vimc.so is signed
[0:13:43.999681326] [882] DEBUG IPAManager ipa_manager.cpp:238 Loaded IPA module '/usr/local/lib/aarch64-linux-gnu/libcamera/ipa_vimc.so'
[0:13:44.000531021] [882]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+2127-5988661c
[0:13:44.001322509] [883] DEBUG Camera camera_manager.cpp:105 Starting camera manager
[0:13:44.008882102] [883] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "hantro-vpu" created from /dev/media2
[0:13:44.009380552] [883] DEBUG DeviceEnumerator device_enumerator_udev.cpp:94 Defer media device /dev/media2 due to 2 missing dependencies
[0:13:44.010951737] [883] DEBUG DeviceEnumerator device_enumerator_udev.cpp:319 All dependencies for media device /dev/media2 found
[0:13:44.011063735] [883] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media2: hantro-vpu
[0:13:44.011852098] [883] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "rkvdec" created from /dev/media0
[0:13:44.012037595] [883] DEBUG DeviceEnumerator device_enumerator_udev.cpp:94 Defer media device /dev/media0 due to 1 missing dependencies
[0:13:44.012787167] [883] DEBUG DeviceEnumerator device_enumerator_udev.cpp:319 All dependencies for media device /dev/media0 found
[0:13:44.012876124] [883] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media0: rkvdec
[0:13:44.013861067] [883] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "rkisp1" created from /dev/media1
[0:13:44.014529557] [883] DEBUG DeviceEnumerator device_enumerator_udev.cpp:94 Defer media device /dev/media1 due to 7 missing dependencies
[0:13:44.017853047] [883] DEBUG DeviceEnumerator device_enumerator_udev.cpp:319 All dependencies for media device /dev/media1 found
[0:13:44.017970004] [883] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media1: rkisp1
[0:13:44.018715201] [883] DEBUG Camera camera_manager.cpp:148 Found registered pipeline handler 'PipelineHandlerIPU3'
[0:13:44.019227943] [883] DEBUG Camera camera_manager.cpp:148 Found registered pipeline handler 'PipelineHandlerRPi'
[0:13:44.019344900] [883] DEBUG Camera camera_manager.cpp:148 Found registered pipeline handler 'PipelineHandlerRkISP1'
[0:13:44.019561605] [883] DEBUG DeviceEnumerator device_enumerator.cpp:311 Successful match for media device "rkisp1"
[0:13:44.020256344] [883] DEBUG V4L2 v4l2_videodevice.cpp:577 /dev/video4[cap]: Opened device platform:rkisp1: rkisp1: rkisp1_stats
[0:13:44.020423758] [883] DEBUG V4L2 v4l2_videodevice.cpp:577 /dev/video5[out]: Opened device platform:rkisp1: rkisp1: rkisp1_params
[0:13:44.020583881] [883] DEBUG V4L2 v4l2_videodevice.cpp:577 /dev/video2[cap]: Opened device platform:rkisp1: rkisp1: rkisp1
[0:13:44.020803794] [883] DEBUG V4L2 v4l2_videodevice.cpp:577 /dev/video3[cap]: Opened device platform:rkisp1: rkisp1: rkisp1
[0:13:44.033270561] [883] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[0:13:44.033462183] [883] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
[0:13:44.041566351] [883] DEBUG IPAManager ipa_manager.cpp:316 IPA module /usr/local/lib/aarch64-linux-gnu/libcamera/ipa_rkisp1.so signature is valid
[0:13:44.043145118] [883] DEBUG Camera camera_manager.cpp:160 Pipeline handler "PipelineHandlerRkISP1" matched
[0:13:44.043320699] [883] DEBUG Camera camera_manager.cpp:148 Found registered pipeline handler 'SimplePipelineHandler'
[0:13:44.043396239] [883] DEBUG Camera camera_manager.cpp:148 Found registered pipeline handler 'PipelineHandlerUVC'
[0:13:44.043431239] [883] DEBUG Camera camera_manager.cpp:148 Found registered pipeline handler 'PipelineHandlerVimc'
Using camera platform/ff110000.i2c/i2c-1/1-0010 ov13850
[0:13:44.044306809] [882] DEBUG Camera camera.cpp:831 streams configuration: (0) 1920x1920-NV12
[0:13:44.044473348] [882]  INFO Camera camera.cpp:890 configuring streams: (0) 1920x1920-NV12
[0:13:44.044818384] [883] DEBUG MediaDevice media_device.cpp:808 /dev/media1[rkisp1]: rkisp1_isp[2] -> rkisp1_resizer_mainpath[0]: 0
[0:13:44.044859800] [883] DEBUG MediaDevice media_device.cpp:808 /dev/media1[rkisp1]: rkisp1_isp[2] -> rkisp1_resizer_selfpath[0]: 0
[0:13:44.044892175] [883] DEBUG MediaDevice media_device.cpp:808 /dev/media1[rkisp1]: ov13850 1-0010[0] -> rkisp1_isp[0]: 0
[0:13:44.044922507] [883] DEBUG RkISP1 rkisp1.cpp:994 Enabling link from sensor 'ov13850 1-0010' to ISP
[0:13:44.044942340] [883] DEBUG MediaDevice media_device.cpp:808 /dev/media1[rkisp1]: ov13850 1-0010[0] -> rkisp1_isp[0]: 1
[0:13:44.044970632] [883] DEBUG MediaDevice media_device.cpp:808 /dev/media1[rkisp1]: rkisp1_isp[2] -> rkisp1_resizer_mainpath[0]: 1
[0:13:44.044991340] [883] DEBUG RkISP1 rkisp1.cpp:670 Configuring sensor with 4224x3136-SBGGR10_1X10
[0:13:44.045042381] [883] DEBUG RkISP1 rkisp1.cpp:676 Sensor configured with 4224x3136-SBGGR10_1X10
[0:13:44.045072422] [883] DEBUG RkISP1 rkisp1.cpp:687 ISP input pad configured with 4032x3024-SBGGR10_1X10 crop (0x0)/4032x3024
[0:13:44.045106546] [883] DEBUG RkISP1 rkisp1.cpp:693 Configuring ISP output pad with 4032x3024-YUYV8_2X8 crop (0x0)/4032x3024
[0:13:44.045132504] [883] DEBUG RkISP1 rkisp1.cpp:705 ISP output pad configured with 4032x3024-YUYV8_2X8 crop (0x0)/4032x3024
[0:13:44.045164295] [883] DEBUG RkISP1 rkisp1_path.cpp:119 Configured main resizer input pad with 4032x3024-YUYV8_2X8 crop (0x0)/4032x3024
[0:13:44.045190253] [883] DEBUG RkISP1 rkisp1_path.cpp:125 Configuring main resizer output pad with 1920x1920-YUYV8_2X8
[0:13:44.045213586] [883] DEBUG RkISP1 rkisp1_path.cpp:143 Configured main resizer output pad with 1920x1920-YUYV8_1_5X8
[0:13:44.055115810] [883] DEBUG V4L2 v4l2_videodevice.cpp:1137 /dev/video2[cap]: 4 buffers requested.
[0:13:44.055699134] [883] DEBUG V4L2 v4l2_videodevice.cpp:1137 /dev/video2[cap]: 0 buffers requested.
[0:13:44.056120002] [882] DEBUG Camera camera.cpp:1029 Starting capture
[0:13:44.056371999] [883] DEBUG V4L2 v4l2_videodevice.cpp:1137 /dev/video5[out]: 4 buffers requested.
[0:13:44.056738910] [883] DEBUG V4L2 v4l2_videodevice.cpp:1137 /dev/video4[cap]: 4 buffers requested.
[0:13:44.057867351] [883] DEBUG V4L2 v4l2_videodevice.cpp:1137 /dev/video2[cap]: 4 buffers requested.
[0:13:44.057951933] [883] DEBUG V4L2 v4l2_videodevice.cpp:1350 /dev/video2[cap]: Prepared to import 4 buffers
[0:13:44.058051973] [883] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
[0:13:44.058107680] [883] ERROR CameraSensor camera_sensor.cpp:569 'ov13850 1-0010': Failed to construct camera sensor info: the camera sensor does not report the active area
[0:13:44.058132472] [883]  WARN RkISP1 rkisp1.cpp:903 Camera sensor information not available
[0:13:44.058393801] [883]  INFO IPARkISP1 rkisp1.cpp:116 Exposure: 4-1756 Gain: 16-248
[0:13:44.058842961] [883] DEBUG Timer timer.cpp:105 Starting timer 0xffff94007638: deadline 0:13:44.063707629
[0:13:44.059075416] [883] DEBUG Event event_dispatcher_poll.cpp:217 timeout 0.004648838
Capture 1 frames
[0:13:44.059809821] [883] DEBUG Event event_dispatcher_poll.cpp:217 timeout 0.003906558
[0:13:44.059904903] [883] DEBUG Event event_dispatcher_poll.cpp:217 timeout 0.003805934
[0:13:44.059936694] [883] DEBUG Event event_dispatcher_poll.cpp:217 timeout 0.003773268
[0:13:44.060244689] [883] DEBUG Event event_dispatcher_poll.cpp:217 timeout 0.003466440
[0:13:44.060354354] [883] DEBUG Event event_dispatcher_poll.cpp:217 timeout 0.003356191
[0:13:44.060456144] [883] DEBUG Event event_dispatcher_poll.cpp:217 timeout 0.003254693
[0:13:44.064427417] [883] DEBUG Timer timer.cpp:105 Starting timer 0xffff94007638: deadline 0:13:44.069589324
[0:13:44.064499749] [883] DEBUG Event event_dispatcher_poll.cpp:217 timeout 0.005093367
[0:13:44.069760168] [883] DEBUG V4L2 v4l2_videodevice.cpp:1439 /dev/video5[out]: Queueing buffer 0
[0:13:44.069908624] [883] DEBUG V4L2 v4l2_videodevice.cpp:1439 /dev/video4[cap]: Queueing buffer 0
[0:13:44.069972790] [883] DEBUG V4L2 v4l2_videodevice.cpp:1439 /dev/video2[cap]: Queueing buffer 0
[0:13:44.070076622] [883] DEBUG V4L2 v4l2_videodevice.cpp:1439 /dev/video5[out]: Queueing buffer 1
[0:13:44.070128829] [883] DEBUG V4L2 v4l2_videodevice.cpp:1439 /dev/video4[cap]: Queueing buffer 1
[0:13:44.070173745] [883] DEBUG V4L2 v4l2_videodevice.cpp:1439 /dev/video2[cap]: Queueing buffer 1
[0:13:44.070248703] [883] DEBUG V4L2 v4l2_videodevice.cpp:1439 /dev/video5[out]: Queueing buffer 2
[0:13:44.070294494] [883] DEBUG V4L2 v4l2_videodevice.cpp:1439 /dev/video4[cap]: Queueing buffer 2
[0:13:44.070336785] [883] DEBUG V4L2 v4l2_videodevice.cpp:1439 /dev/video2[cap]: Queueing buffer 2
[0:13:44.072805705] [883] ERROR V4L2 v4l2_videodevice.cpp:1443 /dev/video2[cap]: Failed to queue buffer 2: Broken pipe
[0:13:44.072883579] [883] DEBUG V4L2 v4l2_videodevice.cpp:1439 /dev/video5[out]: Queueing buffer 3
[0:13:44.072933453] [883] DEBUG V4L2 v4l2_videodevice.cpp:1439 /dev/video4[cap]: Queueing buffer 3
[0:13:44.072978661] [883] DEBUG V4L2 v4l2_videodevice.cpp:1439 /dev/video2[cap]: Queueing buffer 3
[0:13:44.075416957] [883] ERROR V4L2 v4l2_videodevice.cpp:1443 /dev/video2[cap]: Failed to queue buffer 3: Broken pipe
^CExiting
[0:13:46.549173500] [882] DEBUG Camera camera.cpp:1063 Stopping capture
[0:13:46.550491521] [883] DEBUG V4L2 v4l2_videodevice.cpp:1365 /dev/video2[cap]: Releasing buffers
[0:13:46.550872432] [883] DEBUG V4L2 v4l2_videodevice.cpp:1137 /dev/video2[cap]: 0 buffers requested.
[0:13:46.553294103] [883] DEBUG V4L2 v4l2_videodevice.cpp:1365 /dev/video5[out]: Releasing buffers
[0:13:46.553533266] [883] DEBUG V4L2 v4l2_videodevice.cpp:1137 /dev/video5[out]: 0 buffers requested.
[0:13:46.553639723] [883] DEBUG V4L2 v4l2_videodevice.cpp:1365 /dev/video4[cap]: Releasing buffers
[0:13:46.553791387] [883] DEBUG V4L2 v4l2_videodevice.cpp:1137 /dev/video4[cap]: 0 buffers requested.
[0:13:46.562937622] [883] DEBUG V4L2 v4l2_videodevice.cpp:1365 /dev/video3[cap]: Releasing buffers
[0:13:46.563141785] [883] DEBUG V4L2 v4l2_videodevice.cpp:1137 /dev/video3[cap]: 0 buffers requested.
[0:13:46.563723360] [883] DEBUG V4L2 v4l2_videodevice.cpp:1365 /dev/video2[cap]: Releasing buffers
[0:13:46.563854900] [883] DEBUG V4L2 v4l2_videodevice.cpp:1137 /dev/video2[cap]: 0 buffers requested.
[0:13:46.563923732] [883] DEBUG V4L2 v4l2_videodevice.cpp:1365 /dev/video4[cap]: Releasing buffers
[0:13:46.563956398] [883] DEBUG V4L2 v4l2_videodevice.cpp:1137 /dev/video4[cap]: 0 buffers requested.
[0:13:46.564030189] [883] DEBUG V4L2 v4l2_videodevice.cpp:1365 /dev/video5[out]: Releasing buffers
[0:13:46.564065480] [883] DEBUG V4L2 v4l2_videodevice.cpp:1137 /dev/video5[out]: 0 buffers requested.

This seems to work for a while... cam --camera=1 --capture=1 --stream height=1080,width=1920,pixelformat=NV12,role=video

ubuntu@nanopi-m4:~$ uname -ra
Linux nanopi-m4 5.11.0-rc2 #1 SMP PREEMPT Thu Jan 7 23:25:35 -03 2021 aarch64 aarch64 aarch64 GNU/Linux
ubuntu@nanopi-m4:~$ cam --camera=1 --capture=1 --stream height=1080,width=1920,pixelformat=NV12,role=video
[0:09:54.143611045] [853]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+2127-5988661c
[0:09:54.170276236] [854] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[0:09:54.170468152] [854] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
Using camera platform/ff110000.i2c/i2c-1/1-0010 ov13850
[0:09:54.182585068] [853]  INFO Camera camera.cpp:890 configuring streams: (0) 1920x1080-NV12
[0:09:54.192930705] [854] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
[0:09:54.193021705] [854] ERROR CameraSensor camera_sensor.cpp:569 'ov13850 1-0010': Failed to construct camera sensor info: the camera sensor does not report the active area
[0:09:54.193043580] [854]  WARN RkISP1 rkisp1.cpp:903 Camera sensor information not available
[0:09:54.193308994] [854]  INFO IPARkISP1 rkisp1.cpp:116 Exposure: 4-3324 Gain: 16-248
Capture 1 frames
594.389991 (0.00 fps) stream0 seq: 000000 bytesused: 3110400
ubuntu@nanopi-m4:~$ cam --camera=1 --capture=10 --stream height=1080,width=1920,pixelformat=NV12,role=video
[0:10:44.855613219] [856]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+2127-5988661c
[0:10:44.880678921] [857] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[0:10:44.880762045] [857] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
Using camera platform/ff110000.i2c/i2c-1/1-0010 ov13850
[0:10:44.886761567] [856]  INFO Camera camera.cpp:890 configuring streams: (0) 1920x1080-NV12
[0:10:44.896806673] [857] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
[0:10:44.896902338] [857] ERROR CameraSensor camera_sensor.cpp:569 'ov13850 1-0010': Failed to construct camera sensor info: the camera sensor does not report the active area
[0:10:44.896923921] [857]  WARN RkISP1 rkisp1.cpp:903 Camera sensor information not available
[0:10:44.897198085] [857]  INFO IPARkISP1 rkisp1.cpp:116 Exposure: 4-1756 Gain: 16-248
Capture 10 frames
645.091265 (0.00 fps) stream0 seq: 000000 bytesused: 3110400
645.126464 (28.41 fps) stream0 seq: 000001 bytesused: 3110400
645.161664 (28.41 fps) stream0 seq: 000002 bytesused: 3110400
645.196887 (28.39 fps) stream0 seq: 000003 bytesused: 3110400
645.232025 (28.46 fps) stream0 seq: 000004 bytesused: 3110400
645.267241 (28.40 fps) stream0 seq: 000005 bytesused: 3110400
645.302452 (28.40 fps) stream0 seq: 000006 bytesused: 3110400
645.337662 (28.40 fps) stream0 seq: 000007 bytesused: 3110400
645.372923 (28.36 fps) stream0 seq: 000008 bytesused: 3110400
645.478462 (9.48 fps) stream0 seq: 000011 bytesused: 3110400
[0:10:45.590600960] [857] ERROR RkISP1 rkisp1.cpp:310 Can't locate info from frame
avafinger commented 3 years ago

FYI, after the error i get:

cam --camera=1 --capture=1 --file=frame.raw
[0:21:08.535006371] [893]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+2127-5988661c
[0:21:08.561746036] [894] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[0:21:08.561974118] [894] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
Using camera platform/ff110000.i2c/i2c-1/1-0010 ov13850
[0:21:08.579270149] [893]  INFO Camera camera.cpp:890 configuring streams: (0) 1920x1920-NV12
[0:21:08.593634036] [894] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
[0:21:08.593724160] [894] ERROR CameraSensor camera_sensor.cpp:569 'ov13850 1-0010': Failed to construct camera sensor info: the camera sensor does not report the active area
[0:21:08.593746327] [894]  WARN RkISP1 rkisp1.cpp:903 Camera sensor information not available
[0:21:08.594011741] [894]  INFO IPARkISP1 rkisp1.cpp:116 Exposure: 4-1756 Gain: 16-248
Capture 1 frames
[0:21:08.611380063] [894] ERROR V4L2 v4l2_videodevice.cpp:1443 /dev/video2[cap]: Failed to queue buffer 2: Broken pipe
[0:21:08.616915561] [894] ERROR V4L2 v4l2_videodevice.cpp:1443 /dev/video2[cap]: Failed to queue buffer 3: Broken pipe
^CExiting

dmesg|tail

[  499.197454] use of bytesused == 0 is deprecated and will be removed in the future,
[  499.197463] use the actual size instead.
[  499.198072] rkisp1 ff910000.isp0: start pipeline failed -32
[  499.200701] rkisp1 ff910000.isp0: start pipeline failed -32
[  539.974859] rkisp1 ff910000.isp0: start pipeline failed -32
[  539.980695] rkisp1 ff910000.isp0: start pipeline failed -32
[  824.085159] rkisp1 ff910000.isp0: start pipeline failed -32
[  824.087799] rkisp1 ff910000.isp0: start pipeline failed -32
[ 1268.625112] rkisp1 ff910000.isp0: start pipeline failed -32
[ 1268.630721] rkisp1 ff910000.isp0: start pipeline failed -32
initBasti commented 3 years ago

Hi Sebastian,

Hey avafinger,

I am now with my customized kernel with rkisp1 from 5.11 , seems I have broken it.

Can you check for any messages related to the ov13850 driver or the rkisp1 in dmesg?

For the main test, I used the collabora kernel 5.10.rc1 and I was able to stream video with mjpg-streamer (yes, dark and green),

The rkisp1 heavily relies on 3A algorithms feedback loop and the correct setting of the sensor parameters, in order to produce good images. Libcamera is currently able to set the sensor parameters, but there is no implementation of the 3A algorithms feedback loop.

I will have to rebuild it and test again, but this kernel is not good for the NanoPi M4, the board i have here.

I will follow your progress.

Thank you for testing the driver, this will vastly improve the end results.

Is it possible for you to push your kernel tree to GitHub?

I have a fork of the media tree on GitHub: fork. It is currently not on the latest release, as I have been working on libcamera for a few weeks now (+christmas and stuff). I am working on some patches for the OV13850 to fix some issues as we speak and I will update the fork as soon as the patches are ready. I will comment here as soon as I am done.

FYI, after the error i get:


cam --camera=1 --capture=1 --file=frame.raw
[0:21:08.535006371] [893]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+2127-5988661c
[0:21:08.561746036] [894] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[0:21:08.561974118] [894] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
Using camera platform/ff110000.i2c/i2c-1/1-0010 ov13850
[0:21:08.579270149] [893]  INFO Camera camera.cpp:890 configuring streams: (0) 1920x1920-NV12
[0:21:08.593634036] [894] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
[0:21:08.593724160] [894] ERROR CameraSensor camera_sensor.cpp:569 'ov13850 1-0010': Failed to construct camera sensor info: the camera sensor does not report the active area
[0:21:08.593746327] [894]  WARN RkISP1 rkisp1.cpp:903 Camera sensor information not available

All the above errors and warnings are fixed as soon as I have implemented the VIDIOC_SUBDEV_G_SELECTION ioctl.

[0:21:08.594011741] [894] INFO IPARkISP1 rkisp1.cpp:116 Exposure: 4-1756 Gain: 16-248 Capture 1 frames [0:21:08.611380063] [894] ERROR V4L2 v4l2_videodevice.cpp:1443 /dev/video2[cap]: Failed to queue buffer 2: Broken pipe [0:21:08.616915561] [894] ERROR V4L2 v4l2_videodevice.cpp:1443 /dev/video2[cap]: Failed to queue buffer 3: Broken pipe

These two errors appear because of the image format mismatch between the sensor and the image signal processor, you can fix those with this patch for the rkisp1 driver (Adding the 'enumerate frame sizes ioctl' to the driver, will be merged in 5.12) and with this patch for libcamera (handling the image format mismatch, I currently have to handle one more issue, and I am writing the unit tests so it could take 1 or 2 weeks until the hopefully final version is ready).

Using kernel 5.11.rc2 (did not touch rkisp1):

LIBCAMERA_LOG_LEVELS=0 cam --camera=1 --capture=1


LIBCAMERA_LOG_LEVELS=0 cam --camera=1 --capture=1
...
[0:13:44.033270561] [883] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[0:13:44.033462183] [883] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
...
Using camera platform/ff110000.i2c/i2c-1/1-0010 ov13850
[0:13:44.044306809] [882] DEBUG Camera camera.cpp:831 streams configuration: (0) 1920x1920-NV12
[0:13:44.044473348] [882]  INFO Camera camera.cpp:890 configuring streams: (0) 1920x1920-NV12
[0:13:44.044818384] [883] DEBUG MediaDevice media_device.cpp:808 /dev/media1[rkisp1]: rkisp1_isp[2] -> rkisp1_resizer_mainpath[0]: 0
[0:13:44.044859800] [883] DEBUG MediaDevice media_device.cpp:808 /dev/media1[rkisp1]: rkisp1_isp[2] -> rkisp1_resizer_selfpath[0]: 0
[0:13:44.044892175] [883] DEBUG MediaDevice media_device.cpp:808 /dev/media1[rkisp1]: ov13850 1-0010[0] -> rkisp1_isp[0]: 0
[0:13:44.044922507] [883] DEBUG RkISP1 rkisp1.cpp:994 Enabling link from sensor 'ov13850 1-0010' to ISP
[0:13:44.044942340] [883] DEBUG MediaDevice media_device.cpp:808 /dev/media1[rkisp1]: ov13850 1-0010[0] -> rkisp1_isp[0]: 1
[0:13:44.044970632] [883] DEBUG MediaDevice media_device.cpp:808 /dev/media1[rkisp1]: rkisp1_isp[2] -> rkisp1_resizer_mainpath[0]: 1
[0:13:44.044991340] [883] DEBUG RkISP1 rkisp1.cpp:670 Configuring sensor with 4224x3136-SBGGR10_1X10
[0:13:44.045042381] [883] DEBUG RkISP1 rkisp1.cpp:676 Sensor configured with 4224x3136-SBGGR10_1X10
[0:13:44.045072422] [883] DEBUG RkISP1 rkisp1.cpp:687 ISP input pad configured with 4032x3024-SBGGR10_1X10 crop (0x0)/4032x3024

This is the mismatch that I talked about above, here we have a broken pipe.

...

This seems to work for a while... cam --camera=1 --capture=1 --stream height=1080,width=1920,pixelformat=NV12,role=video

ubuntu@nanopi-m4:~$ uname -ra
Linux nanopi-m4 5.11.0-rc2 #1 SMP PREEMPT Thu Jan 7 23:25:35 -03 2021 aarch64 aarch64 aarch64 GNU/Linux
ubuntu@nanopi-m4:~$ cam --camera=1 --capture=1 --stream height=1080,width=1920,pixelformat=NV12,role=video
[0:09:54.143611045] [853]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+2127-5988661c
[0:09:54.170276236] [854] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[0:09:54.170468152] [854] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
Using camera platform/ff110000.i2c/i2c-1/1-0010 ov13850
[0:09:54.182585068] [853]  INFO Camera camera.cpp:890 configuring streams: (0) 1920x1080-NV12
[0:09:54.192930705] [854] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
[0:09:54.193021705] [854] ERROR CameraSensor camera_sensor.cpp:569 'ov13850 1-0010': Failed to construct camera sensor info: the camera sensor does not report the active area
[0:09:54.193043580] [854]  WARN RkISP1 rkisp1.cpp:903 Camera sensor information not available
[0:09:54.193308994] [854]  INFO IPARkISP1 rkisp1.cpp:116 Exposure: 4-3324 Gain: 16-248
Capture 1 frames
594.389991 (0.00 fps) stream0 seq: 000000 bytesused: 3110400
ubuntu@nanopi-m4:~$ cam --camera=1 --capture=10 --stream height=1080,width=1920,pixelformat=NV12,role=video
[0:10:44.855613219] [856]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+2127-5988661c
[0:10:44.880678921] [857] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[0:10:44.880762045] [857] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
Using camera platform/ff110000.i2c/i2c-1/1-0010 ov13850
[0:10:44.886761567] [856]  INFO Camera camera.cpp:890 configuring streams: (0) 1920x1080-NV12
[0:10:44.896806673] [857] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
[0:10:44.896902338] [857] ERROR CameraSensor camera_sensor.cpp:569 'ov13850 1-0010': Failed to construct camera sensor info: the camera sensor does not report the active area
[0:10:44.896923921] [857]  WARN RkISP1 rkisp1.cpp:903 Camera sensor information not available
[0:10:44.897198085] [857]  INFO IPARkISP1 rkisp1.cpp:116 Exposure: 4-1756 Gain: 16-248
Capture 10 frames
645.091265 (0.00 fps) stream0 seq: 000000 bytesused: 3110400
645.126464 (28.41 fps) stream0 seq: 000001 bytesused: 3110400
645.161664 (28.41 fps) stream0 seq: 000002 bytesused: 3110400
645.196887 (28.39 fps) stream0 seq: 000003 bytesused: 3110400
645.232025 (28.46 fps) stream0 seq: 000004 bytesused: 3110400
645.267241 (28.40 fps) stream0 seq: 000005 bytesused: 3110400
645.302452 (28.40 fps) stream0 seq: 000006 bytesused: 3110400
645.337662 (28.40 fps) stream0 seq: 000007 bytesused: 3110400
645.372923 (28.36 fps) stream0 seq: 000008 bytesused: 3110400
645.478462 (9.48 fps) stream0 seq: 000011 bytesused: 3110400
[0:10:45.590600960] [857] ERROR RkISP1 rkisp1.cpp:310 Can't locate info from frame

The last error is very interesting as I have not encountered it before, I will investigate a bit in order to find out how to reproduce it.

Greetings, Sebastian

avafinger commented 3 years ago

Hi Sebastian,

Thanks for the info. Nice work.

Patches applied, kernel 5.11.0-rc2. It is working... :-)

cam --camera=1 --capture=20 --stream height=1080,width=1920,pixelformat=NV12,role=video

[0:17:20.322978750] [2196]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+2127-5988661c-dirty (2021-01-08T14:01:36+00:00)
[0:17:20.349754540] [2197] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 2 on pad 0: Inappropriate ioctl for device
[0:17:20.349885496] [2197] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
Using camera platform/ff110000.i2c/i2c-1/1-0010 ov13850
[0:17:20.362691066] [2196]  INFO Camera camera.cpp:890 configuring streams: (0) 1920x1080-NV12
[0:17:20.377010698] [2197] ERROR V4L2 v4l2_subdevice.cpp:285 'ov13850 1-0010': Unable to get rectangle 1 on pad 0: Inappropriate ioctl for device
[0:17:20.377105196] [2197] ERROR CameraSensor camera_sensor.cpp:569 'ov13850 1-0010': Failed to construct camera sensor info: the camera sensor does not report the active area
[0:17:20.377128238] [2197]  WARN RkISP1 rkisp1.cpp:935 Camera sensor information not available
[0:17:20.377403859] [2197]  INFO IPARkISP1 rkisp1.cpp:116 Exposure: 4-1660 Gain: 16-248
Capture 20 frames
1040.570917 (0.00 fps) stream0 seq: 000000 bytesused: 3110400
1040.604217 (30.03 fps) stream0 seq: 000001 bytesused: 3110400
1040.637498 (30.05 fps) stream0 seq: 000002 bytesused: 3110400
1040.670777 (30.05 fps) stream0 seq: 000003 bytesused: 3110400
1040.704057 (30.05 fps) stream0 seq: 000004 bytesused: 3110400
1040.737337 (30.05 fps) stream0 seq: 000005 bytesused: 3110400
1040.770615 (30.05 fps) stream0 seq: 000006 bytesused: 3110400
1040.803895 (30.05 fps) stream0 seq: 000007 bytesused: 3110400
1040.837174 (30.05 fps) stream0 seq: 000008 bytesused: 3110400
1040.937013 (10.02 fps) stream0 seq: 000011 bytesused: 3110400
1040.970292 (30.05 fps) stream0 seq: 000012 bytesused: 3110400
1041.003571 (30.05 fps) stream0 seq: 000013 bytesused: 3110400
1041.036852 (30.05 fps) stream0 seq: 000014 bytesused: 3110400
1041.070131 (30.05 fps) stream0 seq: 000015 bytesused: 3110400
1041.103402 (30.06 fps) stream0 seq: 000016 bytesused: 3110400
1041.136691 (30.04 fps) stream0 seq: 000017 bytesused: 3110400
1041.169970 (30.05 fps) stream0 seq: 000018 bytesused: 3110400
1041.203248 (30.05 fps) stream0 seq: 000019 bytesused: 3110400
1041.236528 (30.05 fps) stream0 seq: 000020 bytesused: 3110400
1041.269809 (30.05 fps) stream0 seq: 000021 bytesused: 3110400
avafinger commented 3 years ago

Looking forward to seeing your final work. Closing, for now, feel free if you want to reopen so I can test it again.

ov13850