OpenCVFrameGrabber not working in my android project (Error: Could not read frame in start())

[sunnyyssc]

OpenCVFrameGrabber is not working in my project, and I wonder if it is caused by those three .so files shown in the first three warnings (highlighted in bold below). I have already put all the required files (javacpp.jar, javacv.jar, opencv.jar and ffmpeg.jar) in the /libs folder, and added those jar files under the Dependencies setting.

How can I solve this issue?

Logs:

W/linker: .../lib/arm/libjniopencv_imgproc.so: unused DT entry: type 0xf arg 0x42fe8 W/linker: .../lib/arm/libjniopencv_imgcodecs.so: unused DT entry: type 0xf arg 0x5ab7 W/linker:../lib/arm/libjniopencv_videoio.so: unused DT entry: type 0xf arg 0x3c38 W/System.err: org.bytedeco.javacv.FrameGrabber$Exception: read() Error: Could not read frame in start(). W/System.err: at org.bytedeco.javacv.OpenCVFrameGrabber.start(OpenCVFrameGrabber.java:222)

Under dependencies in app's build.gradle:

dependencies { implementation fileTree(include: ['*.jar'], dir: 'libs') implementation 'com.android.support:appcompat-v7:26.1.0' implementation 'com.android.support.constraint:constraint-layout:1.0.2' implementation 'com.android.support:design:26.1.0' testImplementation 'junit:junit:4.12' androidTestImplementation 'com.android.support.test:runner:1.0.1' androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.1' implementation files('libs/javacpp.jar') implementation files('libs/javacv.jar') implementation files('libs/opencv.jar') implementation files('libs/opencv-android-arm.jar') implementation files('libs/opencv-android-x86.jar') implementation files('libs/ffmpeg.jar') implementation files('libs/ffmpeg-android-arm.jar') implementation files('libs/ffmpeg-android-x86.jar') }

[saudet]

OpenCV's video capture doesn't generally work very well on Android. Just use Camera preview if that satisfies your needs: https://github.com/bytedeco/javacv/blob/master/samples/RecordActivity.java

In any case, if you would like to help and get OpenCV to capture better on Android, please let me know and I will help with anything I can! Thanks

[sunnyyssc]

My project actually involves an IP Camera and shows the frames(real time video) on SurfaceView, so I don't think that example will help ??

This is my first time working on IP Camera and JavaCV with Android. If you could help me, that would be sweet :)

[saudet]

Then use either FFmpegFrameGrabber or IPCameraFrameGrabber.

[sunnyyssc]

Tried both grabbers, and the error seemed to be fixed. But I got some other problems.

Since my IP camera stream link doesn't have file extension, I got the exception saying file not found. Then I used my Logitech C270 webcam with Yawcam to see if that would work. However, I got warnings when using those two grabber classes and Yawcam kicked me out from the connection after ~35 sec. I am not so sure why I got those warnings. Below are the logs from both grabbers and the Yawcam console.

Logs when using FFmpegFrameGrabber:

W/linker: .../lib/arm/libjniavutil.so: unused DT entry: type 0xf arg 0x1a944 W/linker: .../lib/arm/libjniswresample.so: unused DT entry: type 0xf arg 0x342d W/linker: .../lib/arm/libjniavcodec.so: unused DT entry: type 0xf arg 0x1cdd0 W/linker: ...//lib/arm/libjniavformat.so: unused DT entry: type 0xf arg 0x19457 W/linker: ...//lib/arm/libjniswscale.so: unused DT entry: type 0xf arg 0x443d W/linker: ...//lib/arm/libjnipostproc.so: unused DT entry: type 0xf arg 0x2ebb W/linker: ...//lib/arm/libjniavfilter.so: unused DT entry: type 0xf arg 0xb0b3 W/linker: ...//lib/arm/libjniavdevice.so: unused DT entry: type 0xf arg 0x4d2d

Logs when using IPCameraFrameGrabber:

W/System.err: org.bytedeco.javacv.FrameGrabber$Exception: The camera stream ended unexpectedly W/System.err: at org.bytedeco.javacv.IPCameraFrameGrabber.grab(IPCameraFrameGrabber.java:167) W/System.err: at com.sunnyyssc.myproject.IPCamera$1.run(IPCamera.java:58) W/System.err: at java.lang.Thread.run(Thread.java:762) W/System.err: Caused by: java.io.EOFException: The camera stream ended unexpectedly W/System.err: at org.bytedeco.javacv.IPCameraFrameGrabber.readImage(IPCameraFrameGrabber.java:221) W/System.err: at org.bytedeco.javacv.IPCameraFrameGrabber.grab(IPCameraFrameGrabber.java:162) W/System.err: ... 2 more

Logs from Yawcam ( 192.168.1.4 is my desktop, 192.168.1.5 is my android device):

[29/1 15:06] + 192.168.1.4 [29/1 15:06] - 192.168.1.4 (Time: 00:00:31, 4922 kB) <-- I manually disconnect [29/1 15:06] + 192.168.1.5 [29/1 15:07] - 192.168.1.5 (Time: 00:00:35, 252 kB) <-- It disconnects automatically [29/1 15:08] + 192.168.1.4 [29/1 15:09] - 192.168.1.4 (Time: 00:01:31, 21398 kB) <-- I manually disconnect [29/1 15:13] + 192.168.1.5 [29/1 15:13] - 192.168.1.5 (Time: 00:00:35, 959 kB) <-- It disconnects automatically [29/1 15:16] + 192.168.1.5 [29/1 15:16] - 192.168.1.5 (Time: 00:00:36, 246 kB) <-- It disconnects automatically [29/1 15:20] + 192.168.1.5 [29/1 15:21] - 192.168.1.5 (Time: 00:00:36, 947 kB) <-- It disconnects automatically

[saudet]

For URLs without extensions, please call setFormat() as per the message of the exception.

Disconnections are a known issue of FFmpeg: https://github.com/bytedeco/javacv/issues/696#issuecomment-324633221 Please report upstream!

[sunnyyssc]

I did use setFormat() just before calling start(). Im not sure if my code is wrong...

Here is my code

OpenCVFrameConverter.ToMat converterToMat = new OpenCVFrameConverter.ToMat();
AndroidFrameConverter androidFrameConverter = new AndroidFrameConverter();

public void initialiseCam(){
        Log.d("Msg", "initialiseCam function called");

        Thread thread = new Thread(new Runnable() {

            @Override
            public void run() {
                try  {
                    IPCameraFrameGrabber ipCameraFrameGrabber = new IPCameraFrameGrabber("http://admin:YWRtaW4=@192.168.1.3:8080/stream/getvideo", 60, 60, TimeUnit.SECONDS);
                    ipCameraFrameGrabber.setFormat("mjpeg");
                    ipCameraFrameGrabber.start();
                    Log.d("Msg", "Frame Grabber started");
                    //Store to mat for other processing
                    mat = converterToMat .convert(ipCameraFrameGrabber.grab());
                    Log.d("Msg", "Converted to mat");
                    //Convert mat back to frame
                    frame = converterToMat .convert(mat);
                    Log.d("Msg", "Converted to frame");
                    bitmap = androidFrameConverter.convert(frame);
                    Log.d("Msg", "Converted to bitmap");
                    ipCameraFrameGrabber.stop();
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        });

        thread.start();
    }

The above code gave these errors:

W/System.err: org.bytedeco.javacv.FrameGrabber$Exception: http://admin:YWRtaW4=@192.168.1.3:8080/stream/getvideo W/System.err: at org.bytedeco.javacv.IPCameraFrameGrabber.start(IPCameraFrameGrabber.java:135) W/System.err: at com.sunnyyssc.myproject.IPCamera$1.run(IPCamera.java:55) W/System.err: at java.lang.Thread.run(Thread.java:762) W/System.err: Caused by: java.io.FileNotFoundException: http://admin:YWRtaW4=@192.168.1.3:8080/stream/getvideo W/System.err: at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:254) W/System.err: at org.bytedeco.javacv.IPCameraFrameGrabber.start(IPCameraFrameGrabber.java:133) W/System.err: ... 2 more

The stream link works on browser, so I don't think the link has problem...??

Also, how about this system error for IPCameraFrameGrabber? It said the stream ended unexpectedly, but I could still see the stream on my browser... Was that possibly be Yawcam causing the problem?

W/System.err: org.bytedeco.javacv.FrameGrabber$Exception: The camera stream ended unexpectedly

[saudet]

It works from your browser, but your browser isn't running on your Android device, right? So you're just having network issues between your server and your device.

[sunnyyssc]

Found the issue! The issue was definitely caused by the link. I changed the other IP camera with different link format, and now I don't have any system error caused by FrameGrabber!

By the way, will these 3 warnings cause problems? Or should I ignore them?

W/linker: .../lib/arm/libjniopencv_core.so: unused DT entry: type 0xf arg 0x896f1 W/linker: .../lib/arm/libjniopencv_imgproc.so: unused DT entry: type 0xf arg 0x42fe8 W/linker: .../lib/arm/libjniopencv_imgcodecs.so: unused DT entry: type 0xf arg 0x5ab7

[saudet]

Great!

Those are just warnings from Android, that's fine.

[sunnyyssc]

Thanks for your help earlier! Now I am able to show the frames from the IP camera to SurfaceView using while loop, and a new thread, however the performance is slow (It is like having ~1-2 second delay when moving from the current frame to the next frame). Also, how can I resize the resolution of a frame? because the frames from the camera are 1280x720. I tried to set ImageHeight and ImageWidth before I start the grabber, but was not working.

EDIT: Just tried to use TimerTask, and now the performance is better, but still slow

[saudet]

Try to use FFmpegFrameGrabber. It's faster and supports resizing.

[sunnyyssc]

When you mean resizing, does it mean I use setImageHeight() and setImageWeight() from FrameGrabber or the ones from FFmpegFrameFilter?

[saudet]

Whichever one you want to use!

[sunnyyssc]

I just tried FFmpegFrameGrabber, and the performance is indeed getting better. I used getFrameRate() to see the frame rate and I get 25fps, but it's still a bit laggy and doesn't look like 25 fps.

[sunnyyssc]

As the video is still a bit slow (low framerate), what else I could do to make it faster? I already set the framerate of my IP camera and the FrameGrabber to 30 fps, but it is still slow. Is it because the camera specs are a bit high?

Camera specs: Coding Format: H.264 Resolution: 1280*720 Frame Rate: 30FPS Mode: VBR Image Quality: Low

[saudet]

lower the resolution? check if your device has a hardware decoder and use that? lots of things to check!

[sunnyyssc]

Unfortunately, apart from Frame Rate and Image Quality, there is nothing I can change in my IP camera settings. Seems like there is nothing I can do on the camera side...

[RojasCarlos98]

Hello, I followed your installation instructions, but in the end I have the following error

More than one file was found with OS independent path 'META-INF/native-image/ios-x86_64/jnijavacpp/reflect-config.json'.

any suggestion?

[saudet]

@RojasCarlos98 Those files are not needed or used by Android, you can exclude them from the build.

[RojasCarlos98]

forgive my ignorance but now I get the following error Cannot fit requested classes in a single dex file (# methods: 100525 > 65536) could you recommend something to me?

[saudet]

This error won't happen with recent versions of Android. Please upgrade your version of Android Studio to the latest version.

[RojasCarlos98]

i am using android 4.1.1

[saudet]

That won't work. Please use Android 5.0+.

[RojasCarlos98]

It is Android studio 4.1.1 the latest stable version, and I am compiling it with compileSdkVersion 30

[saudet]

Well, it works fine for me with Android Studio 4.1.1, so there's something wrong with your installation...

[RojasCarlos98]

according to the manual although I understand it, just add

implementation group: 'org.bytedeco', name: 'javacv-platform', version: '1.5.4'

in the build.gradle it is right?

[saudet]

Yes, that works, but it will include a lot of files that are not used on Android. You can start by reducing that with Gradle JavaCPP as mentioned on this page: https://github.com/bytedeco/javacpp-presets/wiki/Reducing-the-Number-of-Dependencies BTW, this is unrelated to this issue. If you have more questions, please post them somewhere else! I will not be replying to any of your messages here. Thanks

[RojasCarlos98]

thank you very much, really thank you very much