Closed RishabhK12 closed 1 month ago
videoCapture
stays on the old frame after calling onLatestImageAvailable(frame);
?print('frame count: ' + videoCapture.get(cv.CAP_PROP_POS_FRAMES).toString());
always prints 0.0
even setted manually via videoCapture.set(cv.CAP_PROP_POS_FRAMES, frameNum);
?If so, I tested using the following code and it seems every thing is ok.
import 'dart:io';
import 'package:opencv_dart/opencv_dart.dart' as cv;
void main(List<String> args) {
void onLatestImageAvailable(cv.Mat frame) {
print("Working...");
sleep(Duration(seconds: 1));
}
final videoCapture = cv.VideoCapture.create("test/images/small.mp4", apiPreference: cv.CAP_ANY);
double frameNum = 0;
const frameStep = 10;
var (ret, frame) = videoCapture.read();
final frameCount = videoCapture.get(cv.CAP_PROP_FRAME_COUNT);
print("Frame Count: $frameCount");
while (frameNum < frameCount) {
frameNum += frameStep;
videoCapture.read(m:frame);
videoCapture.set(cv.CAP_PROP_POS_FRAMES, frameNum);
onLatestImageAvailable(frame);
print('frame pos: ${videoCapture.get(cv.CAP_PROP_POS_FRAMES)}');
}
print("Finished");
}
If onLatestImageAvailable
is synchronous, theoretically, it should work as expected.
If the problem still exists, please provide a minimum reproducable example so I can try to fix it.
I am running 1.1.0. The VideoCapture moves on but it moves on frame by frame when I use set and when I don't. When I print it out, it always prints to 0.0, same with the frame count and so the code you provided doesn't work. Below I am using a simple widget that allows the user to choose a video and it extracts frames from it.
import 'package:opencv_dart/opencv_dart.dart' as cv;
class VideoDetectorWidget extends StatefulWidget {
const VideoDetectorWidget({super.key});
@override
State<VideoDetectorWidget> createState() => _VideoDetectorWidgetState();
}
class _VideoDetectorWidgetState extends State<VideoDetectorWidget> {
@override
void initState() {
super.initState();
}
void onLatestImageAvailable(cv.Mat frame) {
print("Working...");
sleep(Duration(seconds: 1));
}
void _extractFrames() async {
FilePickerResult? video = await FilePicker.platform.pickFiles(
type: FileType.video,
);
String? videoPath = video?.paths.first;
if (videoPath != null) {
final videoCapture = cv.VideoCapture.create(videoPath, apiPreference: cv.CAP_ANY);
double frameNum = 0;
const frameStep = 10;
var (ret, frame) = videoCapture.read();
final frameCount = videoCapture.get(cv.CAP_PROP_FRAME_COUNT);
print("Frame Count: $frameCount");
while (frameNum < frameCount) {
frameNum += frameStep;
videoCapture.read(m: frame);
videoCapture.set(cv.CAP_PROP_POS_FRAMES, frameNum);
onLatestImageAvailable(frame);
print('frame pos: ${videoCapture.get(cv.CAP_PROP_POS_FRAMES)}');
}
}
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text('Frame Extraction'),
),
body: Center(
child: ElevatedButton(
onPressed: _extractFrames,
child: Text('Extract Frames'),
),
),
);
}
}
@RishabhK12 Confirmed, if you are running on windows or linux, would you please refer to #160 to setup it manually and check whether it is solved?
If you are not running on the above platforms, I have also tested with your code and it also works on windows.
A hotfix version v1.1.0+1 has been published, please upgrade and try again.
@rainyl Thanks for the response. I am running this on android (s23 ultra). The problem is still happening with the same code. I tried both solutions but neither worked. I have found some logs though.
E/cv::error()(15439): OpenCV(4.10.0) Error: Requested object was not found (could not open directory: /data/app/~~huLjj5np-Bk3fmxAWdsKEg==/com.example.lamp-rX3MyqcmJX5DGfbY0Jkrmg==/base.apk!/lib/arm64-v8a) in glob_rec, file /home/runner/work/opencv.full/opencv.full/build/opencv/modules/core/src/glob.cpp, line 279
E/cv::error()(15439): OpenCV(4.10.0) Error: Requested object was not found (could not open directory: /data/app/~~huLjj5np-Bk3fmxAWdsKEg==/com.example.lamp-rX3MyqcmJX5DGfbY0Jkrmg==/base.apk!/lib/arm64-v8a) in glob_rec, file /home/runner/work/opencv.full/opencv.full/build/opencv/modules/core/src/glob.cpp, line 279
E/cv::error()(15439): OpenCV(4.10.0) Error: Requested object was not found (could not open directory: /data/app/~~huLjj5np-Bk3fmxAWdsKEg==/com.example.lamp-rX3MyqcmJX5DGfbY0Jkrmg==/base.apk!/lib/arm64-v8a) in glob_rec, file /home/runner/work/opencv.full/opencv.full/build/opencv/modules/core/src/glob.cpp, line 279
I/flutter (15439): Frame Count: 0.0
@RishabhK12 Finally I figured it out.
TLDR: OpenCV doesn't support it on android. Releated Issues:
Details:
From the opencv source code (v4.10.0 https://github.com/opencv/opencv/blob/71d3237a093b60a27601c20e9ee6c3e52154e8b1/modules/videoio/src/cap_android_mediandk.cpp#L181-L213), we can see that only several props are supported on android mediandk, which is the default backend on android, and I didn't managed to compile opencv with FFMPEG on android so the mediandk is the only backend to process videos on android.
If you or other contributors can successfully compile opencv with FFMPEG (or gstreamer) backend on android, I am very willing to merge it.
Also, on android API > 29 (android 10), the prop CAP_PROP_FRAME_COUNT
was supported, https://github.com/opencv/opencv/blob/71d3237a093b60a27601c20e9ee6c3e52154e8b1/modules/videoio/src/cap_android_mediandk.cpp#L264-L266 , however the opencv used in this project was compile with an API of 24 https://github.com/rainyl/opencv.full/blob/899bc6989070a3cd92eb6bce176aa0fc8e258a9e/profiles/android-armv8#L5 , so CAP_PROP_FRAME_COUNT
is also not supported by this package, it's possible to recompile opencv with an API of 29 but I think it's unnecessary, for keeping compatibility.
Allright, now it's the solution: just use (bool, Mat) read()
, the first return value indicates whether it's success, will return false if reach to the last frame, and use another loop to skip n steps (maybe I will add a parameter step
to read()
in the future). e.g.,
And the output:
BTW, the supported video formats depend on mediandk, I didn't test it but you can refer to https://developer.android.com/media/platform/supported-formats#video-codecs
So if no further problems, I am going to close this issue.
Feel free to reopen it if you still have problems.
Trying to extract every 10 frames from a video to then be run through a neural network using the tflite flutter package. Everything works but the detections displayed are in the wrong spot because of the inference time. They get delayed and are displayed late because the video moves on and it keeps getting further back. Trying to set the VideoCaptures position doesn't work.
Using this to extract frames:
Every 10 frames are extracted and sent through the onLatestImageAvailable method to run a neural network on.
Printing out the value it thinks it is at just produces 0.0. I have tried using CAP_PROP_POS_MSEC and CAP_PROP_POS_AVI_RATIO. This works fine in using opencv in python but trying to incorporate it with a mobile app.