search

radek-k / FFMediaToolkit

FFMediaToolkit is a cross-platform video decoder/encoder library for .NET that uses FFmpeg native libraries. It supports video frames extraction, reading stream metadata and creating videos from bitmaps in any format supported by FFmpeg.
MIT License
352 stars 56 forks source link

Skipped video frames when reading (audio) frame-by-frame are accumulated in memory #69

Closed kskalski closed 3 years ago

kskalski commented 3 years ago

I'm processing large file, but only reading audio frames out of it with media_file_.Audio.TryGetNextFrame(out FFMediaToolkit.Audio.AudioData frame). The process keeps growing the memory usage and finally fails with Encountered error when processing audio: Cannot read next packet from the file Error code: -12 : Cannot allocate memory

Seems like this is because by default we read both video and audio and when user skips the video frames by just using Audio.TryGetNextFrame, then the video frames gets accumulated in internal buffer.

Switching MediaFile.Open(filename) to MediaFile.Open(filename, new MediaOptions() { StreamsToLoad = MediaMode.Audio }) fixes the memory usage. It's not completely clear how the library should handle such unread / undisposed frames, but I guess a reasonable thing to do would be to limit the video frames buffer.

IsaMorphic commented 3 years ago

Yeah, I'd tend to agree with this. Question is more of how the buffer limit would be configured by the user, either via number of frames or by actual memory size.

kskalski commented 3 years ago

number of frames seems to be easier to implement and actually might be also easier to reason about by the user