Playback on Windows skips frames

[goohoobler]

Playback on Windows 11 skips frames - with 4K EXRs , 2K DPXs and even with 1K PNGs. Screen shot shows results after 5 seconds of play - 41 skipped frames (which can be plainly seen as stutters in the played back footage)

This has been attempted on multiple Windows systems with similar results. This has been happening on all the previous builds I have looked at. As the primary duty of software designed to play frame sequences for professional film work , I would expect that this program would be able to play all frames at speed without stuttering or missing frames.

HW : Windows 11 Pro (22H2) Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz 64.0 GB RAM Nvidia 1070 (8GB VRAM)

[ggarra13]

Okay, first, some clarifications.

1) Skipped frames in the HUD is only reliable if you do a straight playback of the sequence, without scrubbing. On v0.9.0 there was a bug that would not reset the skipped frames when going back to the first or last frames of the sequences. In summary, you should not rely too much on the HUD display. 2) Skipped frames for image sequences have to be considered after the caching for them is done. If you play them without the caching having finished you can indeed get stuttering. 3) 4K and higher resolution images can stutter if you play them with the Pixel Toolbar being present, as the pixel color lookup can slow down the OpenGL display. The solution to it is to a) Hide the Pixel Toolbar or b) Switch the Pixel Lookup to "O" (Original) instead of "F" (Full). You can set both or one of those things in the Preferences (F10) if you want to make them work permanently.

2K and 1K images should play without issues with your setup I believe, and you should not see any visual stutters in the playback (you may still get the HUD Skipped Frames info wrong, but that's due to the way mrv2 checks for skipped frames, which is a guess). If you do see stutters visually when playing 2K or 1K images, please upload a .zip with a sample sequence to some cloud service like mega.nz, dropbox, etc. and I'll take a look at it.

[goohoobler]

Thank you for responding. Per your clarifications:

1. No scrubbing was done while playing.
2. Files were loaded into cache before evaluating (green line all the way across frame range).
3. Pixel Toolbar was off. Tried in Full Screen and Presentation mode - same results and was in HD. 4K is worse.

Here is a good test HD EXR sequence uploaded on Wetransfer for 7 days: https://we.tl/t-Z2YOGa7Uf3 Just 2 frames of text alternated for 200 frames. When I play it (after being fully cached) I see pauses and stutters, while the HUD reports skipped frames. Playing at 24fps.

[ggarra13]

Thanks for the sequence. So far, I have bad news, unfortunately.

I tried it on my box with an NVidia 3080 GTX, which is mainly a gamer card, and I get no pauses and no skipped frames with mrv2 v0.9.1 beta, other than the normal flicker due to the refresh rate of the monitor not matching the 24FPS of the sequence.

I realized you told me you have an NVidia 1070, which, I just checked with Bing's AI, was released on June 10th, 2016. So it is quite an old card by today standards. I mistakenly thought it was a pro card and much newer.

mrv2 uses OpenGL 3 for most of its rendering, and your card might not be that optimized for it. OpenGL 3 was released on 2008.

Have you tried some older viewers like the original mrViewer or djv2 to see if they play your sequences better? Those viewers rely on OpenGL 2 and may be more well suited to your graphics card.

You can find the original mrViewer at: https://sourceforge.net/projects/mrviewer/files/v6.2.3/

and djv2 at: https://darbyjohnston.github.io/DJV/download.html

mrViewer also has the skipped frames reporting in the HUD, so you can check it if it also skips frames.

If you find mrViewer or djv2 work better, we can do some further digging into the NVidia driver configuration to see if OpenGL 3 performance can be improved. If they perform just as bad for you as mrv2, I am afraid there's not much that I can do.

[goohoobler]

I have tested with mrV 6.23 , DJV 1.30 and the latest mrV2 version 0.91 beta on sourceforge: Testing at 24 fps and 60fps (to stress test):

mrV 6.23 - plays with skips in windowed mode, but plays almost as expected in Presentation mode (F12)- still a few skips reported but much smoother

mrV2 0.91beta - plays with skips in windowed mode and Presentation mode (no visible difference) - 60fps unusable.

DJV 1.30 - plays as expected in both windowed mode and Full screen mode (u) 24 and 60 fps OK

DJV 1.30 (which has been our goto for years) does what we expect. The older (6.32) mrV is closer in Presentation mode but even then is not quite as stable as DJV.

The 1070 video card is running the latest Studio driver from Nvidia.

[ggarra13]

I'll reach out to @darbyjohnston so that he can share his knowledge about djv_view, DJV2 and tlRender. I think the main issue is your video ram with an OCIO workflow, even if you are not yet fully using it.

djv_view does not support OCIO and only uses a LUT and color transformations, which, my guess, are all done in the CPU by FFmpeg (leaving only one texture in the GPU). That's why you see it perform better and fits nicely in your 1070's VRAM (and you have a fast CPU). Darby can verify if this is the case or not.

I believe DJV2 supported OCIO1 with a 3DLUT like mrViewer, so you might also want to compare it too and see if it is similar to mrViewer. I think DJV2 did everything in its UI with OpenGL so it was also very stressful on the graphics card.

mrViewer 6.23 supports OCIO2 with a 3DLUT, so you have one or two textures if the LUT button is on during playback of sequences, one of which is only an approximation. You can get better performance if you turn off the LUT button. When you play movie files you have 3 textures (for YUV). This I wrote myself so I can verify this is how it works.

mrv2 (and Darby's tlRender library with its not yet released tlplay) supports OCIO2 natively without a LUT. The number of textures used may vary depending on the ICS and Display/View used. In mrv2, the LUT is always on. Darby added a way to turn it off but this is not exposed in mrv2 (so this can be a source of a speed improvement for you). Also, it uses a number of linked shaders, which also adds to the complexity of the viewing pipeline. That's why you need additional VRAM and puts more stress on the GFX card. Also, the timeline is drawn with OpenGL (and worse if you show the thumbnails) which is also another source of slowdown.

[ggarra13]

I asked Darby. He suggested you try the free version of cineSync:

https://www.ftrack.com/en/cinesync/download-cinesync-5#download-cinesync-play

[ggarra13]

I've added automatically turning off OCIO when there's no Input Color Space (ICS) selected (like the default at start) with the released v0.9.1. This might provide some playback improvement, but it may not be enough for you to choose mrv2 over djv_view 1.3.

[goohoobler]

Thank you for the tweaks. While I appreciate the suggestions of other software, I was doing this so that it could improve your program (which has many new features that we desire) and, in general, support truly open source software.

Currently in presentation/Full screen mode (with no input LUT ) playing for 10 seconds mrV2 0.91 - FHD at 24fps the Skipped Frames value is 10 mrV2 - FHD at 60fps the SF value is 100 mrV2 - 4K at 24fps the SF is 50 mrV2 - 4K at 60fps the SF value is 500

Cinesyncplay 5.3.3 - FHD at 24fps plays at reported 24fps but has occasional hiccups Cinesyncplay - FHD at 60fps - only plays at reported 45fps Cinesyncplay - 4K at 24fps plays at reported 24fps but has more occasional hiccups Cinesyncplay - 4K at 60fps - only plays at reported 30fps

DJV 2.08 - FHD at 24fps plays as expected DJV 2.08 - FHD at 60fps plays as expected DJV 2.08 - 4K at 24fps plays at reported 23-25fps and has occasional hiccups DJV 2.08 - 4K at 60fps plays at reported 30fps

Also of note, the hotkey remapping for does not seem to work for the "Quit Program" key assignment , no matter what I change it to , the ESC key will always quit the program (which I don't want, as I am used to using that key to escape from full screen mode) ...

[darbyjohnston]

Interesting tests, I’m surprised DJV 2 fared so well, there’s been an open issue for awhile about playback performance: https://github.com/darbyjohnston/DJV/issues/178

Out of curiosity have you also tested OpenRV and xSTUDIO?

[goohoobler]

First off, Darby, thank you so much for DJV1.3 it has been a crucial part of our studio for many years. I'm not sure any of the new crop currently has performance consistent enough for more than FHD playback with "older" graphics cards... The mods made to mrV2 have helped but at the cost of LUT usage, which is a double edged sword since that is what we need to see footage as intended.. or play with exposure and gamma in real time to evaluate render issues.. I have not tested OpenRV or xSTUDIO because I am not a programmer so can't compile their respective codebases for windows (I can't help but think badly of them for not providing binaries.. but I guess that was a conscious commercial decision on their part...). If you have a lead on where I could get some Windows binaries for them, I would be happy to test them.

Oh, and by the way, the ungendered monarch of open source graphics - Blender - can buffer and play 4K+ footage (in its video editing tab) at a locked in 60fps. However since blender isn't using any hardware trickery in that tab, any color or grading tweak will trigger a rerender at 2fps till cached again.

[ggarra13]

OpenRV can probably be compiled for Windows, but it only supports OCIO 1. I would expect that it should perform in the same range as DJV2 and the old mrViewer. xStudio cannot yet be tested as they have not yet released source code to compile on Windows.

[darbyjohnston]

First off, Darby, thank you so much for DJV1.3 it has been a crucial part of our studio for many years.

I am glad you have found it useful, who are you?

I'm not sure any of the new crop currently has performance consistent enough for more than FHD playback with "older" graphics cards

I appreciate older hardware but sometimes upgrades are necessary. I try and test regularly on hardware even older than your 1070 but it's time consuming with limited resources.

I can't help but think badly of them for not providing binaries.. but I guess that was a conscious commercial decision on their part...

I think they want to provide binaries but are still trying to figure out how.

Performance is important to me and I will continue to work on improving it in tlRender (which mrv2 and cineSync Play are based on). FWIW, from my testing Linux seems to play back smoother than Windows on similar hardware. The smoothest playback I have seen has been with outputting to Blackmagic DeckLink hardware.

[ggarra13]

Thank you for the tweaks. While I appreciate the suggestions of other software, I was doing this so that it could improve your program (which has many new features that we desire) and, in general, support truly open source software.

Much appreciated.

Also of note, the hotkey remapping for does not seem to work for the "Quit Program" key assignment , no matter what I change it to , the ESC key will always quit the program (which I don't want, as I am used to using that key to escape from full screen mode) ...

Find v0.9.2 beta which fixes the hotkeys (Changing any of them did not work, much to my shame :):

.exe https://mega.nz/file/XXREEbra#ExxoY7viXSzJ1g1QAazewI_CEVGjMJLoF0Qoj3HKMz4

.zip https://mega.nz/file/LCJkGCiL#-wR_J1wfK9cXL-23WMhHM7FOqcoqwbjA_05WnqYVYuQ

[goohoobler]

Thanks for the updated version with working hotkey assignments. I'm not sure what else I can do to help either of you improve the consistency of the playback speed for older video cards, but please let me know if there is. I am happy to keep testing as new versions are released. Thanks for all the hard work you are putting in, we really appreciate it.

[ggarra13]

I'm not sure what else I can do to help either of you improve the consistency of the playback speed for older video cards, but please let me know if there is. I am happy to keep testing as new versions are released.

I appreciate all your hard work on testing all the applications.

Thanks to your testing I was able to further improve the performance on Linux by 50% when in windowed mode. Sadly, the optimization does not work on Windows, without getting flickering when dragging the timeline viewport separator bar.

In any case, I give you here a link to that version in case it works for you (also I added reporting of actual FPS, so that you can compare it with cineSync too, since I am curious if I can now match or improve upon its performance):

.exe: https://mega.nz/file/SDImELCa#hLil6pCSYa5iTPZeUtgwaXIJ-dCoqzG1KlXiSYbXuSk

.zip: https://mega.nz/file/3bgTGTiJ#4Hqw8YF-tZl6Sce4mwrOSrXLZWHbNql-gdHn0EEYiTk

[ggarra13]

Besides the previous version, I have another version for you to test but that does not suffer from flickering. I think I managed to get part of the performance improvement I got on Linux also on Windows (at least with my graphics card) when showing the timeline (Full screen does not have an improvement).

.exe: https://mega.nz/file/vXgUXDLY#bjNwscC7k_wjmpceg5qoOi6Pw9WwjdRzYDkaxIE35TY

.zip: https://mega.nz/file/zHg2WbAY#5UMR3wv17lhgLVJemlsTgv54jCRKkwVy-s4j0eeCfWE

Anyway, try these two new versions and let me know if you see some improvement.

[goohoobler]

Thanks for the new builds. The actual FPS indicator is helpful in seeing what is going on. And just a clarification - does it matter if the Pixel Bar is visible in windowed mode (or in fullscreen)- this doesn't seem to make a visible difference, but I am not using an input LUT. Currently in presentation/Full screen mode (with no input LUT , Pixel Bar off) playing for 10 seconds 3x runs: mrV2 0.92beta2 - FHD at 24fps the Skipped Frames value is 7 - speed varies from 22 to 25fps with visible stuttering mrV2 - FHD at 60fps the SF value is 170 - speed varies from 43 to 45fps with visible stuttering mrV2 - 4K at 24fps the SF is 7 - speed varies from 22 to 25fps with visible stuttering mrV2 - 4K at 60fps the SF value is 180 - speed varies from 38 to 42fps with visible stuttering (32-33 windowed) also - FHD/4K at 15fps the Skipped Frames value is 0 but speed varies from 14 to 16fps with visible stuttering but when Input LUT applied: FHD at 24fps the Skipped Frames value is 12 - speed varies from 22 to 25fps with visible stuttering 4K at 24fps the Skipped Frames value is 24 - speed varies from 22 to 25fps with visible stuttering

mrV2 0.92beta3Blit - FHD at 24fps the Skipped Frames value is 6 - speed varies from 22 to 25fps with visible stuttering mrV2 - FHD at 60fps the SF value is 80 - speed varies from 50 to 59fps with visible stuttering mrV2 - 4K at 24fps the SF is 8 - speed varies from 22 to 25fps with visible stuttering mrV2 - 4K at 60fps the SF value is 180 - speed varies from 44 to 48fps with visible stuttering (38-40 windowed)

And just an observation that may mean nothing - the 3D node in the Windows Task manager indicates about 30% utilization full screen (20% windowed) in mrv2, whereas with Blender it is at 50% . The copy node indicates 8% in mrv2 but 0% in Blender ...

Also, sorry if it is obvious, but I cannot find a way to turn off the cursor during playback - is it possible?

[ggarra13]

And just a clarification - does it matter if the Pixel Bar is visible in windowed mode (or in fullscreen)- this doesn't seem to make a visible difference, but I am not using an input LUT.

Yes. The pixel bar can slow it down, particularly at 4K. You should use the new Preferences->Playback->Auto Hide Pixel Bar.

[ggarra13]

mrV2 0.92beta3Blit - FHD at 24fps the Skipped Frames value is 6 - speed varies from 22 to 25fps with visible stuttering mrV2 - FHD at 60fps the SF value is 80 - speed varies from 50 to 59fps with visible stuttering mrV2 - 4K at 24fps the SF is 8 - speed varies from 22 to 25fps with visible stuttering mrV2 - 4K at 60fps the SF value is 180 - speed varies from 44 to 48fps with visible stuttering (38-40 windowed)

I think that's as good as we are going to get. That seems to match or improve upon cineSync's speed.

And just an observation that may mean nothing - the 3D node in the Windows Task manager indicates about 30% utilization full screen (20% windowed) in mrv2, whereas with Blender it is at 50% . The copy node indicates 8% in mrv2 but 0% in Blender ...

Thanks for the info. I was unaware of this information in Task Manager. Regarding Blender, blender's UI is all drawn with OpenGL 2, so, depending on what module you are using, it is not surprising. I am not familiar with blender's playback module. How do I access it so I can try it?

Also, sorry if it is obvious, but I cannot find a way to turn off the cursor during playback - is it possible?

Currently it is not possible. I just ported a feature from mrViewer where the cursor will disappear after 3 seconds of inactivity in Presentation mode. It will be there in the next beta and in v0.9.2 when I release it.

[goohoobler]

Regarding Blender, blender's UI is all drawn with OpenGL 2, so, depending on what module you are using, it is not surprising. I am not familiar with blender's playback module. How do I access it so I can try it?

Sorry if you are a Blender expert, but here are the beginner instructions: In the main window of Blender - Goto the "+" (Add Workspace dropdown) in the top Menu - select Video Editing/Video Editing which should create a new tab . In that tab Goto the sequencer pane and select Add/Image Sequence from the drop down. Select all frames from the sequence (not just the first frame) and enter. This should give you a strip that you can playback. First time through to cache , then it should stabilize playback . You will also probably need to increase the Memory Cache limit to 50000+ In the Preferences - System/Video Sequencer area as well.

I just ported a feature from mrViewer where the cursor will disappear after 3 seconds of inactivity in Presentation mode. It will be there in the next beta and in v0.9.2 when I release it.

Great, Thanks.

[ggarra13]

Sorry if you are a Blender expert, but here are the beginner instructions: In the main window of Blender

Thanks. I am a blender newbie. Got it working. Two questions:

a) How do I bring the view window to full screen (presentation mode) if possible and zoom in/out with hotkeys or with the mouse? b) When I loaded your FullHD test, it played it without problems in the view window (putting mrv2 to shame I admit), but when I loaded a 4K movie it went into a painful "baking" mode. Does it really show 4K movies or does it downres them to the view window size (I lack a 4K monitor to test)?

[goohoobler]

Blender is an amazing program, enjoy your time in it... a) To get presentation mode equivalent is a two step process - First go to the Window menu and select "Toggle Window Fullscreen" (and maybe set a hotkey for it using the Right mouse key then select "Assign Shortcut" - which you can do to most any operation), then hover your mouse cursor over the area you want to maximize (usually the Preview area) and hit Ctrl-Alt-Space (on windows keyboard) to toggle it. b) It will show movies in full resolution but before you load in a 4K+ sequence be sure to change the Scene/Format/Resolution setting in the Properties/Output Tab (The area on the top right of scene - the Printer icon tab) to the exact footage pixels. That should maximize load and cache speeds and provide maximum playback speed. The number keys on the number keypad (if you have an extended keyboard) will zoom the image in discreet amounts - 1=100% , 2=50%, 4=25%, 8=12% and Cntl-2=200%, Cntl-4=400%, Cntl-8=800% also the middle mouse wheel should zoom and middle mouse button should drag/reposition.

[ggarra13]

a) To get presentation mode equivalent

Awkward, but got it working. I just have to get used to it. I loved the hotkey functionality. I've been wanting to add something like that to mrv2 forever. I'll see if I can squeeze it before v1.0.

b) It will show movies in full resolution but before you load in a 4K+ sequence

I don't currently have a 4K sequence, but have a 4K movie (Netflix's Sparks sample movie of 15000 frames at 59.92 FPS). The results with that were not as good as mrv2 on my GPU :). It played the movie fine for some seconds, but it then choked and started dropping fps until it got to a halt. Then it continued at 5 fps or so. I also tried a trailer at HD of 3600 frames at 23.976 and it also had problems playing all the frames without halting every now and then. So it is clear it is CPU bound, not GPU bound where it renders to an offscreen buffer and uses a single quad texture to draw it in OpenGL. The cache just pre-allocates a big buffer and fills it with images until it runs out of memory and then you are in trouble. It uses OCIO2 but it does it all in the CPU. Finally, I tried a multichannel EXR and a multipart EXR (EXR2) and neither could be read (albeit for the sample openexr-images/Beachball I could see the thumbnails, which was odd). For short sequences, as long as you have the RAM, the results are great. But for playing movie files or dealing with EXRs with render passes or OCIO transforms, you have to look elsewhere.