play video with FFmpeg

[YazdanNA]

Hello When I try to use your sample code (JavaFX videoAndAudio) some videos are slow and others are fast, in some videos the sound is jammed and ... Can you help me please?

https://github.com/bytedeco/javacv/blob/master/samples/KazemiFacemarkExample.java

[saudet]

Please try to use this code instead: https://github.com/rladstaetter/javacv-webcam

/cc @rladstaetter

[YazdanNA]

Thank you for your answer But in fact, my problem is that I can not set the frame rate, so some videos play fast and others play slowly, for example, when importing video at 60 frames per second Movie time doubles, and when I play a video at 30 frames per second, the video plays very fast, so if there is a way I can set frame rate manually, I would be very grateful Guide me. I also tried to setFrameRate() but it didn't work. I also used the project you linked to. When I used the webcam, I had no problem. I think it's because of the webcam frame rate, but when I imported the video, I still had the same problem.

And I'm sorry I sent the wrong link above https://github.com/bytedeco/javacv/blob/master/samples/JavaFxPlayVideoAndAudio.java

[saudet]

Ah, I see, what you're looking for is an equivalent to ffmpeg -re, see issue #1633 for at least a way to emulate it in Java.

Ideally, we should add that functionality to FFmpegFrameGrabber. Contributions are welcome!

[YazdanNA]

Actually, I used Thread.sleep () for movies with 30 frames or less, but it does not work for movies with higher frame rates. But thank you anyway.

[jpsacha]

For higher frame rates and certain codecs you may run into situation when grab() takes longer than the frame length. You would need to skip grabs frames to catch up with. You could do that with setTimestamp.

[YazdanNA]

@jpsacha I tried this but it didn't work. Can you give an example?

[jpsacha]

@YazdanNA I created a "simple" example and added to OpenCV_Cookbook examples here: https://github.com/bytedeco/javacv-examples/blob/e5e1ecda675b08c56867358e59cdcbeee82d717e/OpenCV_Cookbook/src/main/scala/opencv_cookbook/chapter11/DisplayVideoSequence.scala

The example aims at playing video at correct playback speed. Delay is added to for slow frame rates. For fast frame rates, some frames are skipped.

A set of stopwatches is used to measure playback speed (playStopWatch). The intention is to match playback time and video frame timestamps. If the playback would get ahead of the frame timestamp a delay is added. If the playback falls behind the frame timestamp a time consuming operation, a frame display or a frame grab, is skipped.

Additional stopwatches are used to continuously measure time taken by the most time consuming operations:

• grabbing frames (grabStopWatch),
• displaying frames (displayStopWatch), and
• skipping frames (setTimestampStopWatch).

They are used to estimate how many frames we need to skip so the playback time can keep up with the frame timestamps.

You can play you own video passing path to the video at the command line. For instance, at SBT command line you could use:

sbt:opencv-cookbook> runMain opencv_cookbook.chapter11.DisplayVideoSequence /path/to/my/video.mp4

[YazdanNA]

@jpsacha Thank you for your time.

[saudet]

@jpsacha Looks good, thanks! It would be great if you could integrate that in FrameGrabber itself, by enhancing what @wangxi761 has already contributed in pull #1659.

[jpsacha]

PR #1659 seem to nicely simulate the "-re" option ("-re" assumes that capture and processing can be done faster that frame rate). Only the Javadoc could be more descriptive how to use it and what the limitations are.

When you need to skip frames things get a bit more complicated, as my example above illustrates. I am not clear if for fast frame rate a frame skipping can be practically implemented using only grabAtFrameRate(). There will need to be a way to also skip processing/display. Maybe by adding a lambda argument for processing/display that grabAtFrameRate could decide to skip. Let me think more about a useful API for that.

[saudet]

PR #1659 seem to nicely simulate the "-re" option ("-re" assumes that capture and processing can be done faster that frame rate). Only the Javadoc could be more descriptive how to use it and what the limitations are.

We could use more docs in general, not just for that. Please feel free to contribute! However, docs also need to be maintained, or they may become inaccurate rather quickly and be less useful than no docs at all...

When you need to skip frames things get a bit more complicated, as my example above illustrates. I am not clear if for fast frame rate a frame skipping can be practically implemented using only grabAtFrameRate(). There will need to be a way to also skip processing/display. Maybe by adding a lambda argument for processing/display that grabAtFrameRate could decide to skip. Let me think more about a useful API for that.

Sure! Sounds good.