search

intel / libxcam

libXCam is a project for extended camera(not limited in camera) features and focus on image quality improvement and video analysis. There are lots features supported in image pre-processing, image post-processing and smart analysis. This library makes GPU/CPU/ISP working together to improve image quality. OpenCL is used to improve performance in different platforms.
Other
592 stars 230 forks source link

What is the status with nvidia support? #650

Closed Tamme closed 5 years ago

Tamme commented 5 years ago

I have been trying image-stitching and test-cl-image executables and they both fail with difficult to debug opencl errors. From one of the issues a while back I noticed that you dont support nvidia cards, has that changed now?

For reference, the error with image-stitching:

----input------ ./test-image-stitching --input "$FOLDER/Camera.000.nv12" --input "$FOLDER/Camera.090.nv12" --input "$FOLDER/Camera.180.nv12" --input "$FOLDER/Camera.270.nv12" --output output.mp4 --input-w 1280 --input-h 800 --output-w 1920 --output-h 640 --scale-mode local --enable-fisheyemap --res-mode 1080p4 --surround-mode bowl --fm false --framerate 30.0 --fisheye-num 4 --save false --loop 1000

------output-----

input0 file: Camera.000.nv12 input1 file: Camera.090.nv12 input2 file: Camera.180.nv12 input3 file: Camera.270.nv12 output file: output.mp4 input width: 1280 input height: 800 output width: 1920 output height: 640 resolution mode: 1080P4 surround mode: bowl view scale mode: local seam mask: false fisheye map: true shading correction: false feature match: false fisheye number: 4 save file: false save topview file: false save freeview file: false framerate: 30.000 loop count: 1000

XCAM WARNING cl_device.cpp:183: cl get device info(CL_DEVICE_IMAGE_PITCH_ALIGNMENT) failed XCAM INFO cl_device.cpp:137: cl get device info, max_compute_unit:20 max_work_item_dims:3 max_work_item_sizes:{1024, 1024, 64} max_work_group_size:1024 image_pitch_alignment:4 XCAM DEBUG cl_device.cpp:57: CL device constructed XCAM DEBUG cl_context.cpp:100: CLContext constructed XCAM DEBUG cl_context.cpp:708: CLCommandQueue constructed XCAM INFO test-image-stitching.cpp:595: calibration config path: /home/lauri/compiles/libxcam/tests/drive2/calib XCAM INFO cl_image_360_stitch.cpp:519: fisheye correction output size width:352 height:640 XCAM WARNING cl_context.cpp:441: create cl image failed, errcode:-40 XCAM WARNING cl_memory.cpp:598: CLImage2D create image 2d failed XCAM WARNING cl_context.cpp:441: create cl image failed, errcode:-40 XCAM WARNING cl_memory.cpp:598: CLImage2D create image 2d failed XCAM WARNING cl_context.cpp:441: create cl image failed, errcode:-40 XCAM WARNING cl_memory.cpp:598: CLImage2D create image 2d failed XCAM WARNING cl_context.cpp:441: create cl image failed, errcode:-40 XCAM WARNING cl_memory.cpp:598: CLImage2D create image 2d failed test-image-stitching: cl_fisheye_handler.cpp:306: virtual XCamReturn XCam::CLFisheyeHandler::prepare_parameters(XCam::SmartPtr&, XCam::SmartPtr&): Assertion `_input[NV12PlaneYIdx].ptr () && _input[NV12PlaneYIdx]->is_valid () && _input[NV12PlaneUVIdx].ptr () && _input[NV12PlaneUVIdx]->is_valid () && _output[NV12PlaneYIdx].ptr () && _output[NV12PlaneYIdx]->is_valid () && _output[NV12PlaneUVIdx].ptr () && _output[NV12PlaneUVIdx]->is_valid ()' failed. ./introduction_real_drive_190723.sh: line 129: 14208 Aborted (core dumped) ./.libs/test-image-stitching --input "$FOLDER/Camera.000.nv12" --input "$FOLDER/Camera.090.nv12" --input "$FOLDER/Camera.180.nv12" --input "$FOLDER/Camera.270.nv12" --output output.mp4 --input-w 1280 --input-h 800 --output-w 1920 --output-h 640 --scale-mode local --enable-fisheyemap --res-mode 1080p4 --surround-mode bowl --fm false --framerate 30.0 --fisheye-num 4 --save false --loop 1000

Thank you!

zongwave commented 5 years ago

libxcam support opencl 2.1 APIs, you need to install opencl driver for Nvidia GPU. (We never try run libxcam on Nvidia device) you can run "test-surround-view --module soft" with CPU version image stitching.

Tamme commented 5 years ago

From here it seems nvidia gpu-s only support opencl 1.2 https://en.wikipedia.org/wiki/OpenCL#OpenCL_2.0_support

clinfo shows that i have: Platform Name Intel(R) CPU Runtime for OpenCL(TM) Applications Number of devices 1 Device Name Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz Device Vendor Intel(R) Corporation Device Vendor ID 0x8086 Device Version OpenCL 2.1 (Build 0) Driver Version 18.1.0.0920 Device OpenCL C Version OpenCL C 2.0 Device Type CPU

It seems that opencl for my cpu is good enough

I'm having difficulties with surround view, so i decided to try image-stitching, which i hoped is middle step to surround view. Is it possible to run that also on CPU?

zongwave commented 5 years ago

please try command parameter: $ ./test-surround-view --help --module soft --dewarp-mode sphere

Tamme commented 5 years ago

You have not implemented the option to do 4 cam surround view in sphere mode. For bowl with 4 cams (side image is black, since i actually dont want 360 view, but have 3 cams totalling ~180 degrees) i get this:

image

And with 3 cam and spehere i get a green image. so have to debug what is wrong there.

Do you think this poor result from intrinsic or extrinsic calibration? Im fairly certain i have done intrinsic correctly. For extrinsic, I have measured the distances, but they are definitely few cm, few degrees off (i have a translation of over a meter for some cameras).

Thanks for your help!

zongwave commented 5 years ago

you can dump fisheye correction images and feature match images by enable "#define XCAM_TEST_STREAM_DEBUG 1" at tests/test_stream.h "#define XCAM_CV_FM_DEBUG 1" at modules/ocv/cv_feature_match.cpp & cv_feature_match_cluster.cpp

Tamme commented 5 years ago

Okay. Got it alot better now. thank you.