[CACHE] RAM spikes and jerky uncached playback

[ThomasWilshaw]

Platform Windows 7 SP1 64 Bit Intel HD Graphics 4600 Olive 0.2.0-6a50c0a6 (Release build) Built with Visual Studio 2019 and vcpkg libraries

First Issue

I've been loking a bit more into why I'm getting that RAM spike when first dropping clips onto the timeline. It seems that a new FFmpegFramePool is being created everytime. Here's the code that does it (ffmpegdecoder.cpp line:94 cachonedemand):

    {
      QMutexLocker map_locker(&instance_map_lock_);      
      // FIXME: Test code, this should be changed later
      FFmpegFramePool* frame_pool = frame_pool_map_.value(stream().get());     
      if (!frame_pool) {
        frame_pool = new FFmpegFramePool(256,
                                         our_instance->stream()->codecpar->width,
                                         our_instance->stream()->codecpar->height,
                                         static_cast<AVPixelFormat>(our_instance->stream()->codecpar->format));
        frame_pool_map_.insert(stream().get(), frame_pool);
      }      our_instance->SetFramePool(frame_pool);
      // End test code
    }

The frame pool it creates is about 3GB, mainly becasue the pool is able to hold 256 frames (1920x1080x3channelsx2charsx256frames ~3GB). If I change the hardcoded 256 to say 8, it massively reduces the RAM usage and doesn't seem to have any adverse effects.

I set the hardcoded number to 128 then added two clips and got two RAM spikes:

Interestingly the second spike trails off here but doesn't always. To make the splikes happen I had to not only clear the cache through Olive but also delete the empty folders in the cache location, this is however realistic as normally one wouldn't be repeatedly reloading the same 3 test clips. The second spike only appears when the timeline cursor is placed on the second clip.

This issue also appears in master.

Second Issue I don't know if these are related so I can add this to seperate report if that's more sensible.

I also looked at the uncached playback and found a possible source of the judder. In FFmpegDecoderInstance::RetrieveFrame line 1138 cacheondemand there's a while loop that we sometimes get stuck in for a while. I graphed how many times each frame goes through the loop and there's some obvious spikes that must cause issues: https://docs.google.com/spreadsheets/d/1ENkPfadBz81ffF3ktp-tS5gMC3eRsDh8JXTjygrrFKQ/edit?usp=sharing

Equally if I add a clip to the timeline and then immediately move the cursor position to the end of the clip this loop is run through approximately the same number of times as there are frames up to that point in the timeline.

[itsmattkc]

Yes, this is all known behavior. The memory pool actually saves a lot of memory since it's always more efficient (and helps against memory fragmentation) to allocate one large block than several small blocks (though as you've noted, ideally the 256 would be configurable or auto-adjusted to each system, hence the FIXME). I think the main difference in cacheondemand is that the auto-cache would usually cache largely sequentially, which cuts down a lot on seeking compared to playing uncached frames (and seeking necessarily consumes more memory).

That while loop is just the retrieve function - it's just the reality of seeking a heavy codec. Uncached playback was never intended to be "good" or even "usable", particularly with heavy codecs. It was intended to experiment with giving users fidelity over when/where the cache occurs and how their system resources are used.

[ThomasWilshaw]

Seeing the FIXME I did wonder if you were already aware of this ;) I suppose the problem is not that large chunks or memory are allocated, it's that there doesn't seem to be any limit on how big it can get. Even if you could manually configure the size, say 128 not 256, you'd still hit a machines RAM limit quite quickly. 10 2k clips would be about 15GB. Either that or the memory needs to be freed more often, not just on closure (or is this already happening but not fast enough for my low spec laptop?). I don't think it's unreasonable someone would dump a load of clips on the timeline and scrub through and look at things before properly caching, could be wrong though :) Apologies if this has already been discussed/in the pipeline.

For the second part that makes sense, thanks for the explanation.

[ThomasWilshaw]

As an example (on caheondemand branch) I drop an Alexa clip onto the timeline, let it cache then drop another on. It immediately crashes my laptop by tryingt to use more memory than it has.