search

avafinger / cap-v4l2

capture frames from CMOS camera ov5640 / ov8865 using V4l2 and OpenCV
18 stars 20 forks source link

does not work with opencv4 #5

Open karu2003 opened 1 year ago

karu2003 commented 1 year ago
cap.c: In function ‘int v4l2_display_pix_format(int)’: cap.c:197:1: warning: no return statement in function returning non-void [-Wreturn-type] 197 } ^ cap.c: In function ‘int v4l2_retrieve_frame(int, int, int, int)’: cap.c:409:5: error: ‘IplImage’ was not declared in this scope 409 IplImage *frame_ipl; ^~~~ cap.c:409:15: error: ‘frame_ipl’ was not declared in this scope 409 IplImage *frame_ipl; ^~~~~ cap.c:410:5: error: ‘CvMat’ was not declared in this scope 410 CvMat frame_bgr; ^~~~~ cap.c:462:27: error: invalid conversion from ‘void’ to ‘unsigned char’ [-fpermissive] 462 frame_yuv = calloc(ALIGN_16B(width) height 3, sizeof(unsigned char)); ~~^~~~~~~~~~~~~~
void*

cap.c:464:9: error: ‘frame_bgr’ was not declared in this scope; did you mean ‘frame_yuv’? 464 | frame_bgr = cvMat(height, width, CV_8UC3, (void ) frame_yuv); | ^~~~~ | frame_yuv cap.c:464:21: error: ‘cvMat’ was not declared in this scope 464 | frame_bgr = cvMat(height, width, CV_8UC3, (void ) frame_yuv); | ^~~~~ cap.c:466:9: error: ‘cvSaveImage’ was not declared in this scope 466 | cvSaveImage(frame_name, &frame_bgr, 0); | ^~~ cap.c: In function ‘int v4l2_close_camera(int, int)’: cap.c:491:1: warning: control reaches end of non-void function [-Wreturn-type] 491 | } | ^

avafinger commented 1 year ago

Try with this headers:

#include <opencv2/core/core.hpp>
#include <opencv2/imgproc/imgproc_c.h>
#include <opencv2/imgproc.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/opencv.hpp>
#include <opencv2/videoio.hpp>
#include <opencv2/highgui.hpp>

Edit:

You need namespace:

using namespace std; using namespace cv;

cv::cvMat

karu2003 commented 1 year ago

I build "cap". Does not work, does not save jpeg. Reports this. pi@bpim2z:~/src/cap-v4l2$ ./cap 1270 720 4 1 -999 -1 -1 ---- cap parameters ----- width: 1270 height: 720 v4l2 buffers: 4 exposure: -999 hflip: -1 vflip: -1 Mode: V4L2_MODE_VIDEO Driver: "sun6i-csi" Card: "sun6i-csi-capture" Bus: "platform:1cb0000.camera" Version: 1.0 Capabilities: a4200001 Input: 0 v4l2: unable to set stream parm. v4l2: failed to init camera.

My init setup. media-ctl --device /dev/media0 --set-v4l2 '"ov5640 1-003c":0[fmt:YUYV8_2X8/1280x720]' v4l2-ctl --set-fmt-video=width=1280,height=720

it works fswebcam --displayfps 1 -S 30 -d /dev/video0 -r 1280x720 -p YUV420P - > cam_1280x720_yuv420p.jpg

karu2003 commented 1 year ago

pi@bpim2z:~/src/cap-v4l2$ media-ctl --device /dev/media0 -p Media controller API version 6.2.2

Media device information

driver sun6i-csi model Allwinner A31 CSI Device serial bus info platform:1cb0000.camera hw revision 0x0 driver version 6.2.2

Device topology

avafinger commented 1 year ago

Try to lower FPS (try 30 or 15) here:

https://github.com/avafinger/cap-v4l2/blob/master/cap.c#L296

    parms.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    parms.parm.capture.capturemode =0;
    parms.parm.capture.timeperframe.numerator = 1;
    parms.parm.capture.timeperframe.denominator = 30; // or 15
    if (-1 == xioctl(fd, VIDIOC_S_PARM, &parms)) {
        CAP_ERROR_RET("unable to set stream parm.");
    }
karu2003 commented 1 year ago

the same! parms.parm.capture.timeperframe.denominator = 7; // or 15 ---- cap parameters ----- width: 1270 height: 720 v4l2 buffers: 4 exposure: -999 hflip: -1 vflip: -1 Mode: V4L2_MODE_VIDEO Driver: "sun6i-csi" Card: "sun6i-csi-capture" Bus: "platform:1cb0000.camera" Version: 1.0 Capabilities: a4200001 Input: 0 v4l2: unable to set stream parm. v4l2: failed to init camera.

avafinger commented 1 year ago

Did you try? parms.parm.capture.timeperframe.denominator = 30;

Then comment the whole code:

if 0

parms.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
parms.parm.capture.capturemode =0;
parms.parm.capture.timeperframe.numerator = 1;
parms.parm.capture.timeperframe.denominator = 30; // or 15
if (-1 == xioctl(fd, VIDIOC_S_PARM, &parms)) {
    CAP_ERROR_RET("unable to set stream parm.");
}

endif

karu2003 commented 1 year ago

:( parms.parm.capture.timeperframe.denominator = 30;

avafinger commented 1 year ago

If you want to grab a JPG image, the sensor (OV5640) is able to deliver it without any conversion (without OpenCV). PR applied.

PanderMusubi commented 1 year ago

See also https://github.com/avafinger/cap-v4l2/pull/7

Unfortunately, I get this error:

./cap 1920 1080 8 1 -999 -1 -1 ---- cap parameters ----- width: 1920 height: 1080 v4l2 buffers: 8 exposure: -999 hflip: -1 vflip: -1 Mode: V4L2_MODE_VIDEO v4l2: doesn't support video capturing. v4l2: failed to init camera.

The system I use is Armbian_23.02.2_Pine64_jammy_current_5.15.93.img with to /boot/armbianEnv.txt the following line added:

camera_type=ov5640

Here is additional info https://pastebin.com/QbRAmxka and https://pastebin.com/PVeY1ef3

Just to be sure, I am using the correct kernel module for this camera? https://pine64.com/product/pine64-5-megapixel-camera-module/?v=0446c16e2e66 Has been a while that I had this working. The s5k4ec has nothing to do with this camera, correct?

karu2003 commented 1 year ago

See also #7

Unfortunately, I get this error:

./cap 1920 1080 8 1 -999 -1 -1 ---- cap parameters ----- width: 1920 height: 1080 v4l2 buffers: 8 exposure: -999 hflip: -1 vflip: -1 Mode: V4L2_MODE_VIDEO v4l2: doesn't support video capturing. v4l2: failed to init camera.

The system I use is Armbian_23.02.2_Pine64_jammy_current_5.15.93.img with to /boot/armbianEnv.txt the following line added:

camera_type=ov5640

Here is additional info https://pastebin.com/QbRAmxka and https://pastebin.com/PVeY1ef3

Just to be sure, I am using the correct kernel module for this camera? https://pine64.com/product/pine64-5-megapixel-camera-module/?v=0446c16e2e66 Has been a while that I had this working. The s5k4ec has nothing to do with this camera, correct?

where is your camera ov5640? Check out lsmod.

and add /etc/modules

PanderMusubi commented 1 year ago

Okay, loaded it manually now. This looks good?

# modprobe ov5640
# lsmod|grep ov5640
ov5640                 32768  0
v4l2_fwnode            24576  1 ov5640
v4l2_async             24576  2 v4l2_fwnode,ov5640
videodev              204800  6 sunxi_cedrus,v4l2_async,ov5640,videobuf2_v4l2,videobuf2_common,v4l2_mem2mem
mc                     49152  6 sunxi_cedrus,videodev,ov5640,videobuf2_v4l2,videobuf2_common,v4l2_mem2mem
# lsmod|grep v4l2
v4l2_fwnode            24576  1 ov5640
v4l2_async             24576  2 v4l2_fwnode,ov5640
v4l2_mem2mem           36864  1 sunxi_cedrus
videobuf2_v4l2         24576  2 sunxi_cedrus,v4l2_mem2mem
videobuf2_common       49152  5 sunxi_cedrus,videobuf2_dma_contig,videobuf2_v4l2,v4l2_mem2mem,videobuf2_memops
videodev              204800  6 sunxi_cedrus,v4l2_async,ov5640,videobuf2_v4l2,videobuf2_common,v4l2_mem2mem
mc                     49152  6 sunxi_cedrus,videodev,ov5640,videobuf2_v4l2,videobuf2_common,v4l2_mem2mem

Also added it to /etc/modules and booting reports

/var/log/syslog:Mar 29 10:38:47 pine20 systemd-modules-load[366]: Inserted module 'ov5640'

Then still

~/cap-v4l2# ./cap 1920 1080 8 1 -999 -1 -1
---- cap parameters -----
width: 1920
height: 1080
v4l2 buffers: 8
exposure: -999
hflip: -1
vflip: -1
Mode: V4L2_MODE_VIDEO
v4l2: doesn't support video capturing.
v4l2: failed to init camera.

To be sure, this is the correct driver for this camera? It used to work with different version of Ubuntu in the past. What are other ways to see what might be the problem? Is the the DTB file in Armbian supporting this?

karu2003 commented 1 year ago

I dont See CSI . I must have 2 Video Dev.

PanderMusubi commented 1 year ago

I don't have a CSI module

# find /lib/modules -name '*csi*'|grep -v scsi|grep -v nfcsim

has no results.

Also, all sun4 modules only are:

# find /lib/modules -name 'sun4*'
/lib/modules/5.15.93-sunxi64/kernel/drivers/iio/adc/sun4i-gpadc-iio.ko
/lib/modules/5.15.93-sunxi64/kernel/sound/soc/sunxi/sun4i-spdif.ko
/lib/modules/5.15.93-sunxi64/kernel/sound/soc/sunxi/sun4i-i2s.ko

I have been looking into the build config. I think this file is used https://github.com/armbian/build/blob/main/config/kernel/linux-sunxi64-current.config#L4479 and has ov5640 as a module but nothing on CSI. Should it have a line like https://github.com/armbian/build/blob/main/config/kernel/linux-sunxi-current.config#L4274 with =m ? Would that provide me with the required module?

(Additionally, I found https://github.com/armbian/build/blob/main/patch/kernel/archive/sunxi-6.1/patches.armbian/arm-dts-overlay-Add-Overlays-for-sunxi.patch#L1205 but I don't think this interferes.)

karu2003 commented 1 year ago

the camera without csi will not work. is there csi in your driver tree? I also use Armbian. I added my overlays. i use Banana Pi M2 Zero.

PanderMusubi commented 1 year ago

Thanks. I'm building Armbian install image with CSI as module. Do you use SUN4I_CSI or SUN6I_CSI?

Where/what are the overlays you are using?

avafinger commented 1 year ago

CONFIG_VIDEO_SUN6I_CSI=m

and also check out how to use it:

https://github.com/avafinger/bananapi-zero-ubuntu-base-minimal#working-with-cameras

PanderMusubi commented 1 year ago

I have the kernel modules not in my build and I can load all of them. How can I check if the device tree overlay https://github.com/avafinger/OV5640_camera/blob/master/sun50i-a64-pine64-plus.dtb is already supported by the Armbian build?

PanderMusubi commented 1 year ago

Ah, just found dtdiff and dtc -I dtb -O dts file.dtb and will dive into it.

karu2003 commented 1 year ago

I have overlay for the Camera OV5640. Only 1 command armbian-add overlay ......

PanderMusubi commented 1 year ago

I did the following:

  1. build armbian edge, installed and booted
  2. wget https://github.com/avafinger/OV5640_camera/raw/master/sun50i-a64-pine64-plus.dtb (I have noticed also a s5k4ec exists, doubled checked that I was using the correct one)
  3. dtc -I dtb -O dts sun50i-a64-pine64-plus.dtb > sun50i-a64-pine64-plus.dts
  4. armbian-add-overlay sun50i-a64-pine64-plus.dts which installs it in /boot/overlay-user/sun50i-a64-pine64-plus.dtbo and adds the line user_overlays=sun50i-a64-pine64-plus to /boot/armbianEnv.txt (in that file is also overlay_prefix=sun50i-a64)
  5. added camera_type=ov5640 to /boot/armbianEnv.txt
  6. added ov5640 to /etc/modules
  7. rebooted and checked lsmod|grep ov56 ov5640 40960 0 v4l2_fwnode 24576 1 ov5640 v4l2_async 24576 2 v4l2_fwnode,ov5640 videodev 204800 5 sunxi_cedrus,v4l2_async,ov5640,videobuf2_v4l2,v4l2_mem2mem mc 53248 7 sunxi_cedrus,v4l2_async,videodev,ov5640,videobuf2_v4l2,videobuf2_common,v4l2_mem2mem
  8. wget https://github.com/PanderMusubi/cap-v4l2/archive/refs/heads/master.zip and unzipped it
  9. ran ./install_deps.sh and ./build_script_A64.sh
  10. installed v4l-utils and ran v4l2-ctl --list-devices cedrus (platform:cedrus): /dev/video0 /dev/media0
  11. so missing the csi device I think, also running ./cap 1270 720 4 1 -999 -1 -1 gives an error ---- cap parameters ----- width: 1270 height: 720 v4l2 buffers: 4 exposure: -999 hflip: -1 vflip: -1 Mode: V4L2_MODE_VIDEO v4l2: doesn't support video capturing. v4l2: failed to init camera.
  12. also build fswebcam and ran ./fswebcam --displayfps 1 -S 30 -d /dev/video0 -r 1280x720 -p YUV420P - > cam_1280x720_yuv420p.jpg --- Opening /dev/video0... Trying source module v4l2... /dev/video0 opened. No input was specified, using the first. Unable to query input 0. VIDIOC_ENUMINPUT: Inappropriate ioctl for device
  13. running media-ctl -p only reports on cedrus
  14. running v4l2-ctl --list-devices give cedrus (platform:cedrus): /dev/video0 /dev/media0

Any ideas how to fix this or make the next step?

avafinger commented 1 year ago

The error reported here: https://github.com/avafinger/cap-v4l2/blob/master/cap.c#L285

That means your camera only supports MPLANE. There is no support for MPLANE in cap-v4l2

Change that part of the code to check if true:

 if (!(caps.capabilities & V4L2_CAP_VIDEO_CAPTURE)) {
     if ((cap.capabilities & V4L2_CAP_VIDEO_CAPTURE_MPLANE))  {
         CAP_ERROR_RET("doesn't not support mplane at the momment.");
     } else {
        CAP_ERROR_RET("doesn't support video capturing.");
    }
 }

I think the same is also true for fswebcam. Maybe installing v4l2loopback would help.

Maybe i will revisit fswebcam and add mplane support if you don't find any suitable version on the net, i can't promise it soon.