Error when passing numpy array to C++ function

[NatanelBirarov]

Hello, I have a task of creating a wrapper to an old C++ project (first time I'm working with C++). Basically I am reading a frame from a RealSense camera and passing it to a function that does some calculations. The frame is passed as a numpy array with dtype=numpy.uint8, and the C++ function accept a parameter of type std::vector<int>. This method works without a problem, however I also tried to read an image using cv2.imread() as passing that array to the function, but this gives this very long error that I don't really know how to figure out:

 *** Break *** segmentation violation
 Generating stack trace...
 0x0000007f5ef12818 in int* std::__copy_move<false, true, std::random_access_iterator_tag>::__copy_m<int>(int const*, int const*, int*) at /usr/include/c++/7/bits/stl_algobase.h:369 from /home/jetson/Desktop/Drone-Nav/Algo/build/bin/main.so
 0x0000007f5ef117e8 in int* std::__copy_move_a<false, int const*, int*>(int const*, int const*, int*) at /usr/include/c++/7/bits/stl_algobase.h:387 from /home/jetson/Desktop/Drone-Nav/Algo/build/bin/main.so
 0x0000007fa3972284 in <unknown function>
 0x0000007fa3972060 in <unknown function>
 0x0000007fa3f560b0 in <unknown> from /usr/local/lib/python3.6/dist-packages/cppyy_backend/lib/libcppyy_backend.so
 0x0000007fa3f57240 in Cppyy::CallConstructor(long, unsigned long, unsigned long, void*) at /tmp/pip-install-yrg55g3z/cppyy-backend_6498151ba3cd4503af6784629c9191c4/src/clingwrapper.cxx:986 from /usr/local/lib/python3.6/dist-packages/cppyy_backend/lib/libcppyy_backend.so
 0x0000007fa3bcdc90 in CPyCppyy::CPPMethod::Execute(void*, long, CPyCppyy::CallContext*) at /tmp/pip-install-5ulqtgw3/cpycppyy_fce813bfa6d0488ab6737b8d02dfe3f7/src/CPPMethod.cxx:905 from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000007fa3bc7b28 in CPyCppyy::CPPConstructor::Call(CPyCppyy::CPPInstance*&, _object*, unsigned long, _object*, CPyCppyy::CallContext*) at /tmp/pip-install-5ulqtgw3/cpycppyy_fce813bfa6d0488ab6737b8d02dfe3f7/src/CPPConstructor.cxx:133 from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000007fa3be94b4 in <unknown> from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000007fa3bf3730 in <unknown> from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000000000608da8 in PyObject_Call + 0x50 from python3
 0x0000007fa3b90728 in <unknown> from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x00000000005bbffc in PyCFunction_Call + 0xa4 from python3
 0x0000007fa3bebbc4 in <unknown> from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000000000608da8 in PyObject_Call + 0x50 from python3
 0x0000000000597e30 in <unknown> from python3
 0x00000000005a740c in <unknown> from python3
 0x0000000000608da8 in PyObject_Call + 0x50 from python3
 0x0000007fa3ba7170 in <unknown> from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000007fa3bce188 in CPyCppyy::CPPMethod::ConvertAndSetArgs(_object*, unsigned long, CPyCppyy::CallContext*) at /tmp/pip-install-5ulqtgw3/cpycppyy_fce813bfa6d0488ab6737b8d02dfe3f7/src/CPPMethod.cxx:873 from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000007fa3bce4c4 in CPyCppyy::CPPMethod::Call(CPyCppyy::CPPInstance*&, _object*, unsigned long, _object*, CPyCppyy::CallContext*) at /tmp/pip-install-5ulqtgw3/cpycppyy_fce813bfa6d0488ab6737b8d02dfe3f7/src/CPPMethod.cxx:930 (discriminator 3) from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000007fa3be9a68 in <unknown> from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000000000607948 in _PyObject_FastCallDict + 0xd0 from python3
 0x000000000052b850 in <unknown> from python3
 0x00000000005306c0 in _PyEval_EvalFrameDefault + 0x4770 from python3
 0x000000000052b108 in <unknown> from python3
 0x0000000000631598 in PyRun_FileExFlags + 0x118 from python3
 0x0000000000636c2c in PyRun_SimpleFileExFlags + 0xfc from python3
 0x0000000000621428 in Py_Main + 0xe78 from python3
 0x0000000000420d3c in main + 0x1cc from python3
 0x0000007fa848a7a0 in __libc_start_main + 0xe0 from /lib/aarch64-linux-gnu/libc.so.6
 0x0000000000420e94 in <unknown> from python3
 *** Break *** segmentation violation
 Generating stack trace...
 0x0000007f5ef12818 in int* std::__copy_move<false, true, std::random_access_iterator_tag>::__copy_m<int>(int const*, int const*, int*) at /usr/include/c++/7/bits/stl_algobase.h:369 from /home/jetson/Desktop/Drone-Nav/Algo/build/bin/main.so
 0x0000007f5ef117e8 in int* std::__copy_move_a<false, int const*, int*>(int const*, int const*, int*) at /usr/include/c++/7/bits/stl_algobase.h:387 from /home/jetson/Desktop/Drone-Nav/Algo/build/bin/main.so
 0x0000007fa3972284 in <unknown function>
 0x0000007fa3972060 in <unknown function>
 0x0000007fa3f560b0 in <unknown> from /usr/local/lib/python3.6/dist-packages/cppyy_backend/lib/libcppyy_backend.so
 0x0000007fa3f57240 in Cppyy::CallConstructor(long, unsigned long, unsigned long, void*) at /tmp/pip-install-yrg55g3z/cppyy-backend_6498151ba3cd4503af6784629c9191c4/src/clingwrapper.cxx:986 from /usr/local/lib/python3.6/dist-packages/cppyy_backend/lib/libcppyy_backend.so
 0x0000007fa3bcdc90 in CPyCppyy::CPPMethod::Execute(void*, long, CPyCppyy::CallContext*) at /tmp/pip-install-5ulqtgw3/cpycppyy_fce813bfa6d0488ab6737b8d02dfe3f7/src/CPPMethod.cxx:905 from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000007fa3bc7b28 in CPyCppyy::CPPConstructor::Call(CPyCppyy::CPPInstance*&, _object*, unsigned long, _object*, CPyCppyy::CallContext*) at /tmp/pip-install-5ulqtgw3/cpycppyy_fce813bfa6d0488ab6737b8d02dfe3f7/src/CPPConstructor.cxx:133 from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000007fa3be94b4 in <unknown> from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000007fa3bf3730 in <unknown> from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000000000608da8 in PyObject_Call + 0x50 from python3
 0x0000007fa3b90728 in <unknown> from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x00000000005bbffc in PyCFunction_Call + 0xa4 from python3
 0x0000007fa3bebbc4 in <unknown> from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000000000608da8 in PyObject_Call + 0x50 from python3
 0x0000000000597e30 in <unknown> from python3
 0x00000000005a740c in <unknown> from python3
 0x0000000000608da8 in PyObject_Call + 0x50 from python3
 0x0000007fa3ba7170 in <unknown> from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000007fa3bce188 in CPyCppyy::CPPMethod::ConvertAndSetArgs(_object*, unsigned long, CPyCppyy::CallContext*) at /tmp/pip-install-5ulqtgw3/cpycppyy_fce813bfa6d0488ab6737b8d02dfe3f7/src/CPPMethod.cxx:873 from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000007fa3bce4c4 in CPyCppyy::CPPMethod::Call(CPyCppyy::CPPInstance*&, _object*, unsigned long, _object*, CPyCppyy::CallContext*) at /tmp/pip-install-5ulqtgw3/cpycppyy_fce813bfa6d0488ab6737b8d02dfe3f7/src/CPPMethod.cxx:930 (discriminator 3) from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000007fa3be9a68 in <unknown> from /usr/local/lib/python3.6/dist-packages/libcppyy.cpython-36m-aarch64-linux-gnu.so
 0x0000000000607948 in _PyObject_FastCallDict + 0xd0 from python3
 0x000000000052b850 in <unknown> from python3
 0x00000000005306c0 in _PyEval_EvalFrameDefault + 0x4770 from python3
 0x000000000052b108 in <unknown> from python3
 0x0000000000631598 in PyRun_FileExFlags + 0x118 from python3
 0x0000000000636c2c in PyRun_SimpleFileExFlags + 0xfc from python3
 0x0000000000621428 in Py_Main + 0xe78 from python3
 0x0000000000420d3c in main + 0x1cc from python3
 0x0000007fa848a7a0 in __libc_start_main + 0xe0 from /lib/aarch64-linux-gnu/libc.so.6
 0x0000000000420e94 in <unknown> from python3

Sometimes it also wrote someone like could not convert argument 1, but not always.

This is the python code:

import cppyy
import os
import sys
import cv2
import numpy as np

cppyy.add_include_path('./Algo/include')
cppyy.include('NewAlgoManager.h')
cppyy.load_library('./Algo/build/bin/main.so')

from cppyy.gbl import NewAlgoManager

algoManager = NewAlgoManager()
image = cv2.imread('testimg.jpg')
image.resize((640, 480, 3))

algoManager.createImageMatrixAndProcessPoints(image, image, image.shape[0], image.shape[1], image.shape[2])

And this is the C++ function:

void NewAlgoManager::createImageMatrixAndProcessPoints(std::vector<int> image1, std::vector<int> image2, int cols, int rows, int channels)
{
    cv::Mat image1Mat;
    cv::Mat image2Mat;
    bool isImage1Mat = createMatFromVec(image1, cols, rows, channels, image1Mat);  //converts std::vector to cv::Mat
    bool isImage2Mat = createMatFromVec(image2, cols, rows, channels, image2Mat);
    if(!isImage1Mat || !isImage2Mat){
        std::cout << "Error creating matrices" << std::endl;
        return;
    }
    else{
        std::cout << "Matrices created successfuly" << std::endl;
    }
    cvtColor(image1Mat, image1Mat, cv::COLOR_BGR2GRAY);  //convert to grayscale
    cvtColor(image2Mat, image2Mat, cv::COLOR_BGR2GRAY);
    images[0] = image1Mat;  //save in array
    images[1] = image2Mat;
    processImagesPoints();
}

Any help to understand what is going on would be appreciated.

Thank you in advance.

[wlav]

The function arguments of std::vector<int> mean that the data is copied every time. That is inefficient, but otherwise (in principle) the best way to prevent memory issues.

The crash itself is in the construction of those std::vector<int> temporaries, though, and there are two possible causes, although both should have been caught by cppyy. 1) that image.resize((640, 480, 3)) may have numpy restructure the memory layout (it shouldn't, though), whereas the std::vector<int> expects flat memory; and 2) uint8 from numpy v.s. int in the vector. The latter is (on most platforms) 4x the size.

I'll have a look, see whether I can reproduce either of these as a problem. In the mean time, an easy workaround would be to change the numpy array of uint8 to one of int32. (This will incur another copy, thus inefficient.)

[wlav]

There was apparently indeed no check on this path (goes through initializer_list) for new-style buffers (as numpy uses). That's now fixed in repo. However, I chose to simply reject this path b/c a generic copy would go through Python for each element conversion, which would be dreadfully slow. Instead, do: image = np.array(image, dtype=np.intc). This will let numpy handle the conversion, which is much faster, and fixes things for the current code as well. (You still incur the other copy to create the std::vector<int> temporary, but that can't be helped: a std::vector can not be used as a view.) Of course, letting the C++ code take a std::vector<uint8_t> argument would save you 1 copy.

[NatanelBirarov]

Hey, thank you very much for the answer, is seems image = np.array(image, dtype=np.intc) indeed fixed the issue! It's still weird to me that reading directly from the camera did not cause this error even though the array was also of type uint8, but as long as both ways work now I'm happy :)

[wlav]

There are a ton of ways of initializing a std::vector: it supports the full tuple test suite. If the type of the data that came directly from the camera wasn't a numpy array (or any array type supporting "new-style" buffers), then it may well have done an element-wise copy, which would promote uint8 to int. (E.g. if not an array type or buffer interface, the code will check for implementation of the sequence interface and use that.)

[wlav]

Closing as presumed clarified.