SokratG / Surround-View

MIT License
149 stars 48 forks source link

Hello, I have a problem. #16

Closed langmanman closed 1 year ago

langmanman commented 1 year ago

the result of my image splicing is correct, SVApp.cpp can see the result of splicing with the following code svtitch->stitch(cameradata, stitch_frame); cv::Mat stitched_frame; stitch_frame.download(stitched_frame); cv::imwrite("/usr/Surround-View-master/data/stitch.jpg", stitched_frame); But execute this code in SVApp.cpp,The result displayed is only the car, no image after stitching bool okRender = dp->render(stitch_frame); IMG_20230707_093254 I put svrent.cpp in auto ok = cuOgl.copyFrom(frame, 0); Change into auto ok = cuOgl.copyFrom(frame, 1); Result display 2023-07-07 09-42-10屏幕截图 Do you know what caused this The opengl configuration is as follows

2023-07-07 09-30-49屏幕截图

SokratG commented 1 year ago

@langmanman Hello! "I put svrent.cpp" what's svrent.cpp? What "stitched_frame" contains when You pass it to render? I mean type of data and steps. I think the problem may be in data type and step of the cv::cuda::GpuMat. Also OpenGL may require a different step value depending on GPU vendor and version. I tested only OpenGL ES 3.2. About: "auto ok = cuOgl.copyFrom(frame, 0); Change into auto ok = cuOgl.copyFrom(frame, 1);" The texture ID in glBindTexture must be 0 if you don't bind any other texture before/after in to current state for rendering.

langmanman commented 1 year ago

Thank you for your reply svrent.cpp refers to svrent.cpp, I wrote it wrong, stitched_frame refers to code svtitch->stitch(cameradata, stitch_frame); After stitching the image stitch_frame, I saved it as jpg picture and found that it was correct, but the rendered result only showed the car without the stitched image. The opengl version I used was also 3.2, is it the problem of nvidia driver? Whether to use nvidia opengl or intel opengl

langmanman commented 1 year ago

The above image is the result of combining four pictures, the car is round under the car, and the result of using four cameras is square under the car 微信图片_20230710175624

SokratG commented 1 year ago

Thank you for your reply svrent.cpp refers to svrent.cpp, I wrote it wrong, stitched_frame refers to code svtitch->stitch(cameradata, stitch_frame); After stitching the image stitch_frame, I saved it as jpg picture and found that it was correct, but the rendered result only showed the car without the stitched image. The opengl version I used was also 3.2, is it the problem of nvidia driver? Whether to use nvidia opengl or intel opengl

Ok. As I said the problem in CUDA-OpenGL interoperability. Your texture which contain in "stitched_frame" not correctly pass to opengl buffer. I need more information about the data in stitched_frame: image The data type and step in your "stitched_frame".

But I'm sure the problem in OpenGL texture representation. I have tested only on OpenGL ES 3.2 not in OpenGL 4.6. I think the problem not in driver of your GPU, on screenshot your drivers is correct. The data may be different stored in OpenGL 4.6.

  1. Check stitched data representation(type and step). You can directly change the step value, because this is public field of cv::cuda::GpuMat
  2. Check CUDA-OpenGL interoperability in here, it may be need add some modifications for OpenGL 4.6.
  3. For testing all render process is correctly work try to modify here FragColor variable with some random color. Also here could be a problem because I remember I tested the bowl render code on my laptop with OpenGL 4.6 and this part mostly suite for OpenGL ES 3.2. You can try to use just "highp vec3 color = vec3(texture(surroundTexture, textCoord));"
  4. I invert texture coordinate for correct render here. This is also needed only for OpenGL ES 3.2
langmanman commented 1 year ago

My opengl version is 4.6 and opengl es version is 3.2. Are both of your versions 3.2 251614175-e4c0fcf2-aa63-4b17-a9e4-8118d69ed247

langmanman commented 1 year ago

Thank you for your reply, the type and step information of stitich_frame is as follows. 252533593-8d169afe-c891-4067-a9a8-0b7829785e1b I changed the following two code, and the result is not changed. (1)highp vec4 color_a = vec4(color, 1.f); I changed 1.f to something else. (2)highp vec3 color = vec3(texture(surroundTexture, textCoord)); 252542544-857eb158-9e7e-4eeb-8c9b-fe29bab6813c I modified the above two code results are unchanged

SokratG commented 1 year ago

The step is strange...What the resolution of your stitched_image? Because it must be equal the width. The type is wrong the 19 it's CV_16S but it should be 16 or CV_8UC3, see here because your prepare opengl buffer with 3 channels by 8 bits (here).

In addition, you misunderstood my suggestions to change part of the code in shader programs, please re-read carefully.

langmanman commented 1 year ago

I set stitch_frame to read an image, and then look at stitich_frame type and step, again only show the car, using the image and the result is as follows

stitch_frame

2023-07-14 10-40-11屏幕截图 As for the opengl version you mentioned, my opengl es version is also 3.2, is it the same with yours?

langmanman commented 1 year ago

I changed the version of opengl, the version information is as follows, and I get the same result, only the car can be seen 2023-07-18 15-33-50屏幕截图

SokratG commented 1 year ago

I think the problem not in version of OpenGL ES. My guess the problem inside CUDA-OpenGL interoperability. You can try store stitched image in host memory and then try to load into OpenGL texture memory here and example how you can do this here. It's just to understand where the problem is.