roi error - Githubissues

uglymie commented 1 year ago

Why do I encounter the following errors after running the program：

OpenCV: terminate handler is called! The last OpenCV error is:
OpenCV(4.5.4) Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::UMat::UMat,

Did multiple threads simultaneously access the same image data？

uglymie commented 1 year ago

This error only occurs under Windows

duchengyao commented 1 year ago

I'll add some logs soon to determine what's going wrong.

duchengyao commented 1 year ago

@uglymie

I updated the code, you can try it and paste the full log.

I guess the error is due to the following reason:

StitchingParamGenerator did not find a suitable feature point under the current parameters, resulting in an incorrect ROI. Please use "opencv/stitching_detailed" to find the correct parameters. (see https://docs.opencv.org/4.8.0/d8/d19/tutorial_stitcher.html)

uglymie commented 1 year ago

I try the new code，and the full log is as follows：

Initializing video capture...
Done. 4 captures initialized.
[SensorDataInterface] Getting new images... Done.
[StitchingParamGenerator] Initializing...
Features in image #1: 5907
Features in image #2: 7816
Features in image #3: 8922
Features in image #4: 9958
Pairwise matching
Initial camera intrinsics #1:
K:
[1994.958804651337, 0, 966.5747848586661;
 0, 1997.106947533427, 534.455429085504;
 0, 0, 1]
R:
[0.56918865, 0.10596249, -0.81535029;
 -0.0026755817, 0.99189407, 0.12703824;
 0.82220244, -0.070127174, 0.5648585]
Initial camera intrinsics #2:
K:
[2013.520549802599, 0, 987.1038021272301;
 0, 2014.746357242567, 472.0284186898871;
 0, 0, 1]
R:
[0.92335153, 0.028398206, -0.38290417;
 0.0065952297, 0.99594074, 0.089768276;
 0.38389912, -0.085413009, 0.91941613]
Initial camera intrinsics #3:
K:
[2004.71890291166, 0, 1005.312488920533;
 0, 2012.445652331155, 529.1976398203952;
 0, 0, 1]
R:
[0.92907584, -0.037084997, 0.36802566;
 -0.0069636367, 0.99303126, 0.11764481;
 -0.36982384, -0.11186375, 0.92234302]
Initial camera intrinsics #4:
K:
[1993.781389449454, 0, 960;
 0, 1993.781389449454, 540;
 0, 0, 1]
R:
[0.54764557, -0.10394418, 0.83022892;
 0.0034747645, 0.99252748, 0.1219718;
 -0.83670306, -0.063912459, 0.54391444]
Camera #1:
K:
[1994.958804651337, 0, 966.5747848586661;
 0, 1997.106947533427, 534.455429085504;
 0, 0, 1]
R:
[0.56918865, 0.10596249, -0.81535029;
 -0.0026755817, 0.99189407, 0.12703824;
 0.82220244, -0.070127174, 0.5648585]
Camera #2:
K:
[2013.520549802599, 0, 987.1038021272301;
 0, 2014.746357242567, 472.0284186898871;
 0, 0, 1]
R:
[0.92335153, 0.028398206, -0.38290417;
 0.0065952297, 0.99594074, 0.089768276;
 0.38389912, -0.085413009, 0.91941613]
Camera #3:
K:
[2004.71890291166, 0, 1005.312488920533;
 0, 2012.445652331155, 529.1976398203952;
 0, 0, 1]
R:
[0.92907584, -0.037084997, 0.36802566;
 -0.0069636367, 0.99303126, 0.11764481;
 -0.36982384, -0.11186375, 0.92234302]
Camera #4:
K:
[1993.781389449454, 0, 960;
 0, 1993.781389449454, 540;
 0, 0, 1]
R:
[0.54764557, -0.10394418, 0.83022892;
 0.0034747645, 0.99252748, 0.1219718;
 -0.83670306, -0.063912459, 0.54391444]
warped_image_scale: 1999.84
roi[1863 x 1056 from (-2865, 2873)]
roi[1827 x 1047 from (-1728, 2860)]
roi[1847 x 1049 from (-199, 2862)]
roi[1861 x 1058 from (1049, 2856)]
[StitchingParamGenerator] Initialized.
[GetReprojParams] projected_image_roi_vect_refined:
[GetReprojParams] roi [0: 1500x1034 from (0, 0)]
[GetReprojParams] roi [1: 1315x1034 from (363, 13)]
[GetReprojParams] roi [2: 1399x1034 from (149, 11)]
[GetReprojParams] roi [3: 1562x1034 from (299, 17)]
[SetParams] Setting params...
[CreateWeightMap] Creating weight map...
[CreateWeightMap] Creating weight map... Done.
[SetParams] Setting params... Done.
[SensorDataInterface] Getting new images... Done.
[WarpImages] Warping images 0 of 4...
[WarpImages] Warping images 2[WarpImages] Warping images 3 of 4...
[WarpImages] Remapping 3:4 ...
[WarpImages] Warping images 1 of 4...
[WarpImages] Remapping 1:4 ...
 of 4...
[WarpImages] Remapping 2:4 ...
[WarpImages] Remapping 0:4 ...
[WarpImages] Remapped 1:4 ...
[WarpImages] Remapped 2:4 ...
OpenCV: terminate handler is called! The last OpenCV error is:

duchengyao commented 1 year ago

@uglymie

Use the debug mode (or GDB) to see the line where the error was reported, thanks

uglymie commented 1 year ago

OK，here is the debug mode output：

......
[SensorDataInterface] Getting new images... Done.
[WarpImages] Warping images 0 of 4...[WarpImages] Warping images 3 of 4...
[WarpImages] Remapping 3:4 ...
[WarpImages] Warping images 1 of 4...
[WarpImages] Remapping 1:4 ...
[ INFO:25] global C:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\ocl.cpp (973) cv::ocl::OpenCLExecutionContext::Impl::getInitializedExecutionContext OpenCL: initializing thread execution context

[WarpImages] Remapping 0:4 ...
[WarpImages] Warping images 2[ INFO:27] global C:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\ocl.cpp (973) cv::ocl::OpenCLExecutionContext::Impl::getInitializedExecutionContext OpenCL: initializing thread execution context
 of 4...
[WarpImages] Remapping 2:4 ...
[ INFO:26] global C:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\ocl.cpp (973) cv::ocl::OpenCLExecutionContext::Impl::getInitializedExecutionContext OpenCL: initializing thread execution context
[ INFO:28] global C:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\ocl.cpp (973) cv::ocl::OpenCLExecutionContext::Impl::getInitializedExecutionContext OpenCL: initializing thread execution context
[ INFO:29] global C:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\ocl.cpp (973) cv::ocl::OpenCLExecutionContext::Impl::getInitializedExecutionContext OpenCL: initializing thread execution context
[WarpImages] Remapped 3:4 ...
OpenCV(4.5.4) Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::UMat::UMat, file C:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\umatrix.cpp, line 801
[WarpImages] Remapped 0:4 ...
OpenCV: terminate handler is called! The last OpenCV error is:
[WarpImages] Warped images 0 of 4. (7e-07;0.0027361;3e-07OpenCV(4.5.4) Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::UMat::UMat, file C:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\umatrix.cpp, line 801
;362.687)
[WarpImages] Remapped 1:4 ...
OpenCV(4.5.4) Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::UMat::UMat, file C:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\umatrix.cpp, line 801
[WarpImages] Remapped 2:4 ...
OpenCV(4.5.4) Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::UMat::UMat, file C:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\umatrix.cpp, line 801
[RecordVideos] recorded frame 0.
[RecordVideos] recorded frame 1.
[RecordVideos] recorded frame 2.
[RecordVideos] recorded frame 3.
......

duchengyao commented 1 year ago

@uglymie I'm a little confused...

Please use branch test-assertion-failed or follow this commit to log more variable.

BTW:

The multi line code block in markdown use three backticks ( `` ) not single.

```
aaa
```

not

`
aaaaa
`

uglymie commented 1 year ago

sorry, I will try to solve this problem on my own first and communicate with you later.

duchengyao commented 1 year ago

Thanks ^_^

uglymie commented 1 year ago

我还是用中文回复吧 @duchengyao 我目前使用 test-assertion-failed 分支进行测试；在 image_stitcher.cc 130行 cout 的基础上添加了 tmp_umatvect[img_idx].size() 的打印输出：

std::cout << "[test 1] "
              << img_idx << ": "
              << tmp_umat_vect_[img_idx].size()
              << roi_vect_[img_idx].x << ", "
              << roi_vect_[img_idx].y << ", "
              << weightMap_[0].cols << ", "
              << weightMap_[0].rows << std::endl;

最后的log为：

Initializing video capture...
Done. 4 captures initialized.
[SensorDataInterface] Getting new images... Done.
[StitchingParamGenerator] Initializing...
num_img_: 4
[resolution][1920 x 1080]
Features in image #1: 5907
Features in image #2: 7816
Features in image #3: 8922
Features in image #4: 9958
Pairwise matching
Initial camera intrinsics #1:
K:
[1994.958804651337, 0, 966.5747848586661;
 0, 1997.106947533427, 534.455429085504;
 0, 0, 1]
R:
[0.56918865, 0.10596249, -0.81535029;
 -0.0026755817, 0.99189407, 0.12703824;
 0.82220244, -0.070127174, 0.5648585]
Initial camera intrinsics #2:
K:
[2013.520549802599, 0, 987.1038021272301;
 0, 2014.746357242567, 472.0284186898871;
 0, 0, 1]
R:
[0.92335153, 0.028398206, -0.38290417;
 0.0065952297, 0.99594074, 0.089768276;
 0.38389912, -0.085413009, 0.91941613]
Initial camera intrinsics #3:
K:
[2004.71890291166, 0, 1005.312488920533;
 0, 2012.445652331155, 529.1976398203952;
 0, 0, 1]
R:
[0.92907584, -0.037084997, 0.36802566;
 -0.0069636367, 0.99303126, 0.11764481;
 -0.36982384, -0.11186375, 0.92234302]
Initial camera intrinsics #4:
K:
[1993.781389449454, 0, 960;
 0, 1993.781389449454, 540;
 0, 0, 1]
R:
[0.54764557, -0.10394418, 0.83022892;
 0.0034747645, 0.99252748, 0.1219718;
 -0.83670306, -0.063912459, 0.54391444]
Camera #1:
K:
[1994.958804651337, 0, 966.5747848586661;
 0, 1997.106947533427, 534.455429085504;
 0, 0, 1]
R:
[0.56918865, 0.10596249, -0.81535029;
 -0.0026755817, 0.99189407, 0.12703824;
 0.82220244, -0.070127174, 0.5648585]
Camera #2:
K:
[2013.520549802599, 0, 987.1038021272301;
 0, 2014.746357242567, 472.0284186898871;
 0, 0, 1]
R:
[0.92335153, 0.028398206, -0.38290417;
 0.0065952297, 0.99594074, 0.089768276;
 0.38389912, -0.085413009, 0.91941613]
Camera #3:
K:
[2004.71890291166, 0, 1005.312488920533;
 0, 2012.445652331155, 529.1976398203952;
 0, 0, 1]
R:
[0.92907584, -0.037084997, 0.36802566;
 -0.0069636367, 0.99303126, 0.11764481;
 -0.36982384, -0.11186375, 0.92234302]
Camera #4:
K:
[1993.781389449454, 0, 960;
 0, 1993.781389449454, 540;
 0, 0, 1]
R:
[0.54764557, -0.10394418, 0.83022892;
 0.0034747645, 0.99252748, 0.1219718;
 -0.83670306, -0.063912459, 0.54391444]
warped_image_scale: 1999.84
roi[1863 x 1056 from (-2865, 2873)]
roi[1827 x 1047 from (-1728, 2860)]
roi[1847 x 1049 from (-199, 2862)]
roi[1861 x 1058 from (1049, 2856)]
[StitchingParamGenerator] Initialized.
[GetReprojParams] projected_image_roi_vect_refined:
[GetReprojParams] roi [0: 1500x1034 from (0, 0)]
[GetReprojParams] roi [1: 1315x1034 from (363, 13)]
[GetReprojParams] roi [2: 1399x1034 from (149, 11)]
[GetReprojParams] roi [3: 1562x1034 from (299, 17)]
[SetParams] Setting params...
[CreateWeightMap] Creating weight map...
[CreateWeightMap] Creating weight map... Done.
[SetParams] Setting params... Done.
[SensorDataInterface] Getting new images... Done.
[WarpImages] Warping images 0[WarpImages] Warping images 1 of 4...
[WarpImages] Remapping 1:4 ...
[WarpImages] Warping images 2 of 4...
[WarpImages] Remapping 2:4 ...
 of 4...
[WarpImages] Remapping 0:4 ...
[WarpImages] Warping images 3 of 4...
[WarpImages] Remapping 3:4 ...
[WarpImages] Remapped 3:4 ...
[test 1] 3: [1861 x 1058]299, 17, 100, 1080
[WarpImages] Remapped 2:OpenCV: terminate handler is called! The last OpenCV error is:
4OpenCV(4.5.4) Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::UMat::UMat, file C:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\umatrix.cpp, line 801

很明显，tmp_umatvect 的高度是小于1080的，所以出现了roi的错误，但是不知道为什么在Ubuntu系统中就没有这个问题；我发现，你在 stitching_param_generater.cc 的 InitUndistortMap() 函数中从参数文件中读取了 resolution 变量，为[1920x1080]，再使用OpenCV的 initUndistortRectifyMap 函数计算 undist_xmapvector 和 undist_ymapvector ，此时resolution 的值起到了关键作用；如果把 resolution 的高度改小一些，比如 [1280x720]，或者把CreateWeightMap函数的高度设置的小一些，程序就可以成功运行，只不过输出的图像大小不正常。

另外如果只输入两路视频，不管是在Windows或在Ubuntu系统下都会出现以上问题。

duchengyao commented 1 year ago

另外如果只输入两路视频，不管是在Windows或在Ubuntu系统下都会出现以上问题。

@uglymie 果然是这样，我来看看是啥情况>_<

KristinSun commented 1 year ago

OpenCV(4.4.0) Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::UMat::UMat, file C:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\umatrix.cpp, line 547 这个错是怎么解决的？

KristinSun commented 1 year ago

为什么运行程序后会遇到如下错误：

OpenCV: terminate handler is called! The last OpenCV error is:
OpenCV(4.5.4) Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::UMat::UMat,

多个线程是否同时访问同一个图像数据？

解决了吗这个问题?

duchengyao commented 1 year ago

为什么运行程序后会遇到如下错误：
OpenCV: terminate handler is called! The last OpenCV error is:
OpenCV(4.5.4) Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::UMat::UMat,
多个线程是否同时访问同一个图像数据？
解决了吗这个问题?

老婆生孩子了，工作又比较忙，没时间看了，抱歉

SHL2023 commented 11 months ago

image_stitcher.cc 45行原代码：CreateWeightMap(undist_ymap_vector[0].rows, blend_width); 改为：CreateWeightMap(roivect[0].height, 150); roi越界问题即可得到解决

SHL2023 commented 11 months ago

image_stitcher.cc 45行原代码：CreateWeightMap(undist_ymap_vector[0].rows, blend_width); 改为：CreateWeightMap(roivect[0].height, 150); roi越界问题即可得到解决

可能是因为CreateWeightMap使用的[0]的参数不适用于其他图片，调整参数值至合理的区间或者创建一个WeightMapVector为每张图片创建合适的WeightMap

duchengyao commented 11 months ago

image_stitcher.cc 45行原代码：CreateWeightMap(undist_ymap_vector[0].rows, blend_width); 改为：CreateWeightMap(roivect[0].height, 150); roi越界问题即可得到解决

可能是因为CreateWeightMap使用的[0]的参数不适用于其他图片，调整参数值至合理的区间或者创建一个WeightMapVector为每张图片创建合适的WeightMap

新年快乐

我去看一下，谢谢^_^

LoveChina-3000 commented 9 months ago

我也遇到这个情况，windows下： [WarpImages] Remapped 1:4 ... [ INFO:16@85.723] global ocl.cpp:984 cv::ocl::OpenCLExecutionContext::Impl::getInitializedExecutionContext OpenCL: initializing thread execution context

OpenCV(4.8.0) Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::UMat::UMat, file D:\opencv\opencv-4.8.0\modules\core\src\umatrix.cpp, line 801 OpenCV(4.8.0) Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::UMat::UMat, file D:\opencv\opencv-4.8.0\modules\core\src\umatrix.cpp, line 801 OpenCV: terminate handler is called! The last OpenCV error is: OpenCV(4.8.0) Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::UMat::UMat, file D:\opencv\opencv-4.8.0\modules\core\src\umatrix.cpp, line 801 [WarpImages] Remapped 3:4 ... OpenCV(4.8.0) Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::UMat::UMat, file D:\opencv\opencv-4.8.0\modules\core\src\umatrix.cpp, line 801 [RecordVideos] recorded frame 1. [RecordVideos] recorded frame 2.

duchengyao commented 9 months ago

Thanks to @SHL2023 .

I have update code I updated the code and fixed the out of range error.

If you have any questions, please reopen this issue.

duchengyao / gpu-based-image-stitching

roi error #10