Prioritize media loading

[egnor]

Right now, all media loading is done in parallel, which isn't ideal and can result in unnecessary dropped frames (observed by @aubilenon).

In an ideal world:

• high priority: media frames that will definitely be displayed soon
• medium priority: media frames that will be displayed later
• lower priority: media frames that are being loaded speculatively?
• lower priority: media frames that will be "displayed" but with zero opacity?

But, speculative loads (pins) should probably be able to specify their relative priority (based on likelihood of being used)?

This is all a little tricky:

• it's not clear how to balance priority based on time-to-display and likelihood-of-display
• it's not clear what the best mix of parallel and sequential loading is to use hardware most efficiently
• right now, each frame loader runs in its own thread, they would need to coordinate (or maybe change threading model)
• under the covers, libav (ffmpeg) and the kernel do their own scheduling of loading & decoding activity
• right now, all load requests are collapsed into a set of media-segments-of-interest; priority will need to be added in

For reference, this is my understanding of thread usage in libav: http://ffmpeg.org/pipermail/libav-user/2022-February/012969.html (Note that nobody replied, so I have no idea how accurate this is.)

[egnor]

Strawperson proposal:

• make time-to-display the basic measure of priority for loading a particular frame
• figure out how to capture that in FrameRequest / IntervalSet? <= may be tricky in the general case!
• make likelihood-of-display a constant that gets added to (OR multiplied to??) time-to-display (so you can add a priority penalty of 0.2s to a pin, meaning, start this only after you've gotten the next 0.2s of "real" media loaded), so the true priority of a load becomes adjusted-time-to-display
• have some sort of cross-thread prioritizer that decides who goes and who holds in the media decoder threads, allowing anyone within some fuzz (0.05s?) of the currently highest priority (lowest adjusted-time-to-display) to run
• let libav do whatever libav does internally; the above prioritizer is just gating our get-the-next-frame request calls
• do not mess with actual kernel level thread priority, that way lies madness
• come up with some simple stress tests (along the lines of rickroll_drop.py and jellyfish_buffer.py)