Android crashed when calling CascadeClassifier.detectMultiScale(matGray, facesRect)

runzedong commented 5 years ago

Overview

I'm doing face detection on Android side. The application is supposed to read from a video file and do face detection on frames.

Device

Google Pixel 2

Artifacts Used

    implementation 'org.bytedeco:javacv:1.4.4'
    implementation 'org.bytedeco.javacpp-presets:opencv:4.0.1-1.4.4:android-arm'
    implementation 'org.bytedeco.javacpp-presets:ffmpeg:4.1-1.4.4:android-arm'

Questions

I'm using FFmpegFrameGrabber to extract frames, and convert frames to OpenCV mat with OpenCVFrameConverter.

But the app will crash at the last step when detecting faces on each mat by calling detectMultiScale method.

It complains OpenCL function is not available when calling opencl_check_fn from opencv-4.0.1/modules/core/src/opencl/runtime/opencl_core.cpp:326.

Does this mean the libs are not configured properly in this project or it's because the device doesn't support OpenCL, which indicates JavaCV relies on OpenCL supported devices?
And the directory shown from the log confuses me as well cause it tried to look up the directory home/travis/build/javacpp-presets. But I don't know where this dir come from? Travis CI?
I also tried an OpenCV sample android application on the same device and it works well even though it calls the same method but import from org.opencv.objdetect.CascadeClassifier. So it feels like the error is JavaCV specific? Link of the sample is here

Error log

The error log posted at below:

2019-01-28 14:36:46.259 1267-1267 A/libc: /buildbot/src/android/ndk-release-r18/external/libcxx/../../external/libcxxabi/src/abort_message.cpp:73: abort_message: assertion "terminating with uncaught exception of type cv::Exception: OpenCV(4.0.1) /home/travis/build/javacpp-presets/opencv/cppbuild/android-arm/opencv-4.0.1/modules/core/src/opencl/runtime/opencl_core.cpp:326: error: (-220:Unknown error code -220) OpenCL function is not available: [clGetPlatformIDs] in function 'opencl_check_fn'
    " failed
2019-01-28 14:36:46.259 1267-1267 A/libc: Fatal signal 6 (SIGABRT), code -6 (SI_TKILL) in tid 1267 (ls.samplejavacv), pid 1267 (ls.samplejavacv)

Sample Code

    val faceVideoFilePath = "sdcard/Download/face_tracking_test.mp4" //the test video
    val videoGrabber = FFmpegFrameGrabber(faceVideoFilePath) //init the frame grabber
    val converterToMat = OpenCVFrameConverter.ToMat() //init the frame2mat converter
    val faceDetector: opencv_objdetect.CascadeClassifier // create the CascadeClassifier
    var videoFrame: Frame
    var videoMat: Mat

    while(true){
                videoFrame = videoGrabber.grab()
                if (videoFrame == null) break

                videoMat = converterToMat.convert(videoFrame)

                val videoMatGray = Mat()
                cvtColor(videoMat, videoMatGray, COLOR_BGRA2GRAY)

                val faces = opencv_core.RectVector()
                faceDetector.detectMultiScale(videoMatGray, faces) 

               // may draw rect on face if get faces result return
    }

runzedong commented 5 years ago

ok. so I downgrade the artifact to use OpenCV 3.4.3, and this error is dismissed and no crash again. But I'd like to remain it open and curious about it's not compatible with 4.0.1.

saudet commented 5 years ago

It should still work without OpenCL, but let me check this out here.

BTW, would you have an updated version of FacePreview? If you do, please send a pull request! Thanks

runzedong commented 5 years ago

Just give a bit update. I checked out the latest version with 64-bit builds and it works as expected. So I will close this issue. Updated gradle is below:

    implementation 'org.bytedeco:javacv:1.4.4'
    implementation 'org.bytedeco.javacpp-presets:opencv:4.0.1-1.4.4:android-arm64'
    implementation 'org.bytedeco.javacpp-presets:ffmpeg:4.1-1.4.4:android-arm64'

saudet commented 5 years ago

I was able to reproduce something like that in the emulator. It looks like the C++ runtime crashes on the exception thrown when checking for OpenCL, something that happens only on 32-bit ARM for some reason as explained at https://github.com/android-ndk/ndk/issues/785. In any case, thanks for reporting! Now, we'll need to fix this...

bytedeco / javacv