search

intel / libxcam

libXCam is a project for extended camera(not limited in camera) features and focus on image quality improvement and video analysis. There are lots features supported in image pre-processing, image post-processing and smart analysis. This library makes GPU/CPU/ISP working together to improve image quality. OpenCL is used to improve performance in different platforms.
Other
591 stars 229 forks source link

test surround view falied #589

Closed Vankeee closed 5 years ago

Vankeee commented 5 years ago

Here is my command:

test-surround-view --module soft --input0 1.yuv --input1 2.yuv --input2 3.yuv --input3 4.yuv --output output.mp4 --in-w 640 --in-h 540 --out-w 1920 --out-h 640 --topview-w 1280 --topview-h 720 --scale-mode dualconst --frame-mode multi --save true --save-topview true --loop 1

I ran it on ubuntu 16.04， opencv 3.3.1

It turned out that ：

input0 file: 1.yuv input1 file: 2.yuv input2 file: 3.yuv input3 file: 4.yuv output file: output.mp4 input width: 640 input height: 540 output width: 1920 output height: 640 topview width: 1280 topview height: 720 scaling mode: dualconst frame mode: multiframe save output: true save topview: true loop count: 1 OpenCV: FFMPEG: tag 0x34363258/'X264' is not supported with codec id 28 and format 'mp4 / MP4 (MPEG-4 Part 14)' OpenCV: FFMPEG: fallback to use tag 0x00000021/'!???' XCAM INFO test-surround-view.cpp:620: calibration config path:./calib_params/ XCAM WARNING calibration_parser.cpp:201: open intrinsic file(./calib_params//intrinsic_camera_front.txt) failed. XCAM ERROR test-surround-view.cpp:152: parse intrinsic params (./calib_params//intrinsic_camera_front.txt)failed. XCAM ERROR test-surround-view.cpp:625: parse fisheye dewarp info(idx:0) failed.

What‘s wrong？ Sorry, I am a newbee.

brmarkus commented 5 years ago

Have you installed (either build by source or installing the libraries) the X264 encoder? See e.g. here: "https://trac.ffmpeg.org/wiki/Encode/H.264" and "https://trac.ffmpeg.org/wiki/CompilationGuide". Your example want to stitch multiple raw input streams together and encode/compress it into a MP4 container with an embedded h.264 compressed video stream - and misses the corresponding X264 encoding libraries.

zongwave commented 5 years ago

@Vankeee you need to put your cameras' (front, right, rare and left) intrinsic & extrinsic calibration files under the test folder, use ocamcalib toolbox to do camera calibration: https://sites.google.com/site/scarabotix/ocamcalib-toolbox

pennluo commented 5 years ago

@Vankeee Currently, the camera calibration data only supports the file(format as txt) exported by Matlab's toolbox ocamcalib-toolbox, so if you use it directly, you need to use this toolbox to calibrate.

pennluo commented 5 years ago

@zongwave I got Calibration files calib_results_front.txt ,as below:

polynomial coefficients for the DIRECT mapping function (ocam_model.ss in MATLAB). These are used by cam2world

5 -2.612034e+02 0.000000e+00 1.111171e-03 -1.681827e-06 3.971074e-09

polynomial coefficients for the inverse mapping function (ocam_model.invpol in MATLAB). These are used by world2cam

11 467.821131 299.560088 -35.564686 32.560224 34.181503 -53.088032 93.007996 315.194851 281.555355 109.639155 16.277912

center: "row" and "column", starting from 0 (C convention)

238.809993 480.777383

affine parameters "c", "d", "e"

1.000433 0.000121 0.000871

image size: "height" and "width"

540 960

then I renamed calib_results_front.txt to intrinsic_camera_front.txt What is the data format in extrinsic_camera_front.txt，Can you give some sample files? Thanks!

Vankeee commented 5 years ago

@brmarkus Here is my ffmpeg version information: ffmpeg -version ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609 configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv libavutil 54. 31.100 / 54. 31.100 libavcodec 56. 60.100 / 56. 60.100 libavformat 56. 40.101 / 56. 40.101 libavdevice 56. 4.100 / 56. 4.100 libavfilter 5. 40.101 / 5. 40.101 libavresample 2. 1. 0 / 2. 1. 0 libswscale 3. 1.101 / 3. 1.101 libswresample 1. 2.101 / 1. 2.101 libpostproc 53. 3.100 / 53. 3.100

You see the bold part, did I install the X264 encoder?

zongwave commented 5 years ago

@pennluo extrinsic_camera_front.txt record the camera's position and rotation information as below: ocamcalib-toolbox calibration contains extrinsic results in "RRfin" field.

extrinsic fisheye camera calibration, positions in millimeters, rotations in degrees

Position x

xxxx

Position y

yyyy

Position z

zzzz

Rotation x (roll)

r

Rotation x (pitch)

p

Rotation x (yaw)

y

brmarkus commented 5 years ago

@brmarkus Here is my ffmpeg version information: ffmpeg -version ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609 configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv libavutil 54. 31.100 / 54. 31.100 libavcodec 56. 60.100 / 56. 60.100 libavformat 56. 40.101 / 56. 40.101 libavdevice 56. 4.100 / 56. 4.100 libavfilter 5. 40.101 / 5. 40.101 libavresample 2. 1. 0 / 2. 1. 0 libswscale 3. 1.101 / 3. 1.101 libswresample 1. 2.101 / 1. 2.101 libpostproc 53. 3.100 / 53. 3.100

You see the bold part, did I install the X264 encoder?

Maybe this helps: https://stackoverflow.com/questions/34024041/writing-x264-from-opencv-3-with-ffmpeg-on-linux (looks like the codec-id's do not match; looks like your x264 installation is fine)

pennluo commented 5 years ago

@pennluo extrinsic_camera_front.txt record the camera's position and rotation information as below: ocamcalib-toolbox calibration contains extrinsic results in "RRfin" field.

extrinsic fisheye camera calibration, positions in millimeters, rotations in degrees

Position x

xxxx

Position y

yyyy

Position z

zzzz

Rotation x (roll)

r

Rotation x (pitch)

p

Rotation x (yaw)

y

Thanks!

Vankeee commented 5 years ago

@brmarkus I used "!_writer.open (_file_name, 0x21, 30, frame_size))" to replace "!_writer.open (_file_name, cv::VideoWriter::fourcc ('X', '2', '6', '4'), 30, frame_size)", and the ffmpeg problem solved.

I still got "XCAM ERROR test-surround-view.cpp:625: parse fisheye dewarp info(idx:0) failed." Do you know what is fisheye dewarp info? It's in the intrinsic_camera_front.txt file?

pennluo commented 5 years ago

@Vankeee
A few days ago,I met the same question at compiling,not modify any code,I compiled successfully My solving method: 1.Make sure content format of calibration files(intrinsic&extrinsic) are correct 2.Checking path of calibration files,you can put it under "tests" folder may be you can try

pennluo commented 5 years ago

@brmarkus @zongwave Hi "RRfin" Field gives the pose of the camera relative to the checkerboard in each of the calibration images. I want to use "RRfin" Field to get R(yaw)R(pitch)R(roll)T(x)T(y)T(z) of extrinsic_camera_XXX.txt
here is my thoughts: Before starting calibration. I got 10 calibration images(img 0~9） (img0）------ checkerboard placed on ground (z=0) (img1-img9)--------checkerboad placed any 3D plane After camera calibrate. we can get RRfin matrix 3x3x10 double then I convert 3x3 RRfin[ ; ;1] to R(yaw)R(pitch)R(roll)T(x)T(y)T(z) I wonder if I am right and how can I convert extrinsic matrix to Euler degree？ thanks！

Vankeee commented 5 years ago

@pennluo Now I met another problem.I am using opencv.3.3.1, ubuntu 16.04. Do you use ubuntu? If you use ubuntu, then you should compile 'autofindconer'. Here is the description: _For Linux, please download the Windows version of the toolbox. To make use of the automatic checkerboard extraction, download and compile the following code: autoCornerFinderlinux.zip

When I compiled，it turned out that : **cvcalibinit3.cpp:34:37: fatal error: opencv2/core/internal.hpp: No such file or directory

include <opencv2/core/internal.hpp>**

I searched some, it maybe something about opencv version.

zongwave commented 5 years ago

@pennluo

How do I convert a rotation matrix to Euler angles? http://web.archive.org/web/20041029003853/http://www.j3d.org/matrix_faq/matrfaq_latest.html#Q37

Vankeee commented 5 years ago

@pennluo I gave up using matlab in ubuntu, I changed to Windows. Thanks!

pennluo commented 5 years ago

@zongwave @brmarkus Thanks a lot！

pennluo commented 5 years ago

@Vankeee
Calibration ： Windows + Ocamcalib-toolbox Compiling ： Ubuntu 16.04 +OpenCV3.4

Vankeee commented 5 years ago

@pennluo Could you leave your wechat? or QQ? Wanna chat with you about it= =, is it ok? We can delete the comment later after I add u.

wangqiang624731186 commented 5 years ago

@Vankeee can you share your extrinsiccamera.txt and calibresults.txt ?

Saaloni commented 2 years ago

HI @Vankeee . an you share your extrinsiccamera.txt, calibresults.txt and RRfin?