FPS Performance Issues using ScreenGear and WriteGear

aza547 commented 2 years ago

Hi, first off thanks for this library, it's been really interesting to explore.

The issue I have is that I'm unable to get desireable FPS using the combination of ScreenGear and WriteGear. I want to record my screen at a reasonable FPS for playback (anything above 25 would be fine). I'm recording in 1080p.

However the using the code from the usage example under WriteGear I find that I can only write at about 10 FPS. I was using the default of 25 FPS for WriteGear which resulted in my videos playing back as if they were being fast forwarded.

Question

I have a good computer as far as I'm aware and I'm writing to an SSD. Is this low performance a suprise? It appears the write step is taking the longest

What is the correct way to ensure that recording FPS and video FPS are the same? I want real-time video playback in an app I'm writing so videos that don't match up 1:1 in time spent recording / time spent playing is a problem for me. My idea is to use a combination of cv2.waitKey to "slow down" the recording to match a targeted FPS which can then be passed to WriteGear.

That doesn't solve my performance problems but would allow me to create video at a fixed FPS within my performance limit. I think the example in WriteGear usage doesn't do this and I've not found anything in the docs to assist but this feels like a major thing, so maybe I'm missing something obvious?

I did some analysis by timing each step in the write loop. The below snippet was consistently the profile of where the time was spent.

read time: 0.0005109310150146484 time spent in "frame = stream.read()" write time: 0.03965616226196289 time spent in "writer.write(frame)" total time: 0.04016709327697754

As shown the write is taking the majority of the time. Is it essential the write is done in this loop? I could imagine a structure where we pass the frame to another thread with some enforcement of the ordering.

Acknowledgment

[X] I have searched the issues for my issue and found nothing related or helpful.
[X] I have read the FAQs.
[x] I have read the Documentation.

Context

I'm writing a python app that records screen and performs some analysis on the video before adding timestamps. This is causing my videos to play back in ~2/3rds of the real duration and skew my timestamps.

Your Environment

VidGear version: 0.2.3
Branch: Master
Python version: Python 3.9.7
PiP version: 21.3.1
Operating System and version: Windows 10

Thanks again, Alex

welcome[bot] commented 2 years ago

Thanks for opening this issue, a maintainer will get back to you shortly!

In the meantime:

Read our Issue Guidelines, and update your issue accordingly. Please note that your issue will be fixed much faster if you spend about half an hour preparing it, including the exact reproduction steps and a demo.
Go comprehensively through our dedicated FAQ & Troubleshooting section.
For any quick questions and typos, please refrain from opening an issue, as you can reach us on Gitter community channel.

abhiTronix commented 2 years ago

What is the correct way to ensure that recording FPS and video FPS are the same?

Use constant framerate in compression mode.

However the using the code from the usage example under WriteGear I find that I can only write at about 10 FPS. I was using the default of 25 FPS for WriteGear which resulted in my videos playing back as if they were being fast forwarded.

That's not true, WriteGear do not set any default framerate in compression mode, it let FFmpeg handle it. If you want to change it you need to manually specify it as above.

If you're talking about Non-compression mode then, then yes it has 25 framerate but it doesn't use FFmpeg.

@alexanderkershaw I think you're not using compression mode with FFmpeg backend, rather you're using Non-compression mode with OpenCV backend which is very slow. Can I see the code you're using and terminal log to debug what's wrong?

aza547 commented 2 years ago

Thanks for the quick response. You are correct I'm using non-compression mode. My understanding was that this would not have a performance effect but just cause larger video files - sounds like that was a bad assumption!

I think I initially did try to use compression mode but it didn't work for me straight away so I swapped to non-compression (I don't really care about video size at this stage). I'll definetly try using compression mode now.

I've attatched my recorder code, terminal log below (although it just confirms what you've said above).

PS C:\checkouts\arena-recorder> python .\src\main.py
Interface :: DEBUG :: Started watching: WoWCombatLog-123.txt
Interface :: DEBUG :: Detected ARENA_MATCH_START event
Interface :: DEBUG :: It's 3v3 in Hook Point
13:27:53 ::   WriteGear    :: CRITICAL :: Compression Mode is disabled, Activating OpenCV built-in Writer!
Interface :: INFO :: Started recording
OpenCV: FFMPEG: tag 0x47504a4d/'MJPG' is not supported with codec id 7 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x7634706d/'mp4v'
Interface :: DEBUG :: Detected ARENA_MATCH_END event
Interface :: INFO :: Stopped recording
i: 237
Interface :: DEBUG :: Stopped watching: WoWCombatLog-123.txt

Recorder.txt

abhiTronix commented 2 years ago

Thanks for the quick response. You are correct I'm using non-compression mode. My understanding was that this would not have a performance effect but just cause larger video files - sounds like that was a bad assumption!

Yup @alexanderkershaw That's too much slow to do real-time processing, and will be removed from vidgear in future.

I think I initially did try to use compression mode but it didn't work for me straight away

That's weird. Since you're on windows, it should've work for you straightaway. Are you not using internet when running script ?, since it automatically downloads FFmpeg static binaries from our dedicated server.

abhiTronix commented 2 years ago

Can you try again with compression_mode=True, I've changed something?

abhiTronix commented 2 years ago

@alexanderkershaw It's working fine on my windows 10 machine:

➜ py j.py
19:24:33 ::   WriteGear   ::  DEBUG   :: Compression Mode is enabled therefore checking for valid FFmpeg executable.
19:24:33 ::   WriteGear   ::  DEBUG   :: Output Parameters: {'-vcodec': 'libxvid'}
19:24:33 ::    Helper     ::  DEBUG   :: FFmpeg Windows Download Path: C:\Users\abhiu\AppData\Local\Temp
19:24:33 ::    Helper     ::  DEBUG   :: No Custom FFmpeg path provided. Auto-Installing FFmpeg static binaries from GitHub Mirror now. Please wait...
100%|███████████████████████████████████████████████████████████████████████████████| 105M/105M [00:19<00:00, 5.43MB/s]
19:24:56 ::    Helper     ::  DEBUG   :: Extracting executables.
19:25:00 ::    Helper     ::  DEBUG   :: FFmpeg binaries for Windows configured successfully!
19:25:00 ::    Helper     ::  DEBUG   :: Final FFmpeg Path: C:\Users\abhiu\AppData\Local\Temp\ffmpeg-static-win64-gpl/bin/ffmpeg.exe
19:25:01 ::    Helper     ::  DEBUG   :: FFmpeg validity Test Passed!
19:25:01 ::    Helper     ::  DEBUG   :: Found valid FFmpeg Version: `b'N-104425-g9df3f147f5-20211025'` installed on this system
19:25:01 ::   WriteGear   ::  DEBUG   :: Found valid FFmpeg executable: `C:\Users\abhiu\AppData\Local\Temp\ffmpeg-static-win64-gpl/bin/ffmpeg.exe`.
19:25:01 ::   WriteGear   ::  DEBUG   :: Compression Mode with FFmpeg backend is configured properly.
19:25:01 ::   WriteGear   ::  DEBUG   :: InputFrame => Height:720 Width:1280 Channels:3
19:25:01 ::   WriteGear   ::  DEBUG   :: Executing FFmpeg command: `C:\Users\abhiu\AppData\Local\Temp\ffmpeg-static-win64-gpl/bin/ffmpeg.exe -y -f rawvideo -vcodec rawvideo -s 1280x720 -pix_fmt bgr24 -i - -vcodec libxvid -qscale:v 3 D:\test\Output.mp4`
ffmpeg version N-104425-g9df3f147f5-20211025 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10-win32 (GCC) 20210408
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libvmaf --enable-vulkan --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libglslang --enable-libgme --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20211025
  libavutil      57.  7.100 / 57.  7.100
  libavcodec     59. 12.100 / 59. 12.100
  libavformat    59.  8.100 / 59.  8.100
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8. 15.100 /  8. 15.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Input #0, rawvideo, from 'pipe:':
  Duration: N/A, start: 0.000000, bitrate: 552960 kb/s
  Stream #0:0: Video: rawvideo (BGR[24] / 0x18524742), bgr24, 1280x720, 552960 kb/s, 25 tbr, 25 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> mpeg4 (libxvid))
Output #0, mp4, to 'D:\test\Output.mp4':
  Metadata:
    encoder         : Lavf59.8.100
  Stream #0:0: Video: mpeg4 (mp4v / 0x7634706D), yuv420p(tv, progressive), 1280x720, q=2-31, 200 kb/s, 25 fps, 12800 tbn
    Metadata:
      encoder         : Lavc59.12.100 libxvid
19:25:06 ::   WriteGear   ::  DEBUG   :: Terminating WriteGear Processes.kbits/s speed=1.09x
frame=  132 fps= 27 q=3.0 Lsize=    3370kB time=00:00:05.24 bitrate=5268.7kbits/s speed=1.07x
video:3369kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.043252%

aza547 commented 2 years ago

Was a while ago when I tried - just retried and it's worked fine with compression mode!

PS C:\checkouts\arena-recorder> python .\src\main.py
Interface :: DEBUG :: Started watching: WoWCombatLog-123.txt
Interface :: DEBUG :: Detected ARENA_MATCH_START event
Interface :: DEBUG :: It's 3v3 in Hook Point
13:44:56 ::     Helper     ::  DEBUG  :: No Custom FFmpeg path provided. Auto-Installing FFmpeg static binaries from GitHub Mirror now. Please wait...
  0%|                                                                                       | 0.00/105M [00:00<?, ?B/s]Interface :: DEBUG :: Detected ARENA_MATCH_END event
100%|███████████████████████████████████████████████████████████████████████████████| 105M/105M [00:11<00:00, 9.05MB/s]
13:45:09 ::     Helper     ::  DEBUG  :: Extracting executables.
13:45:11 ::     Helper     ::  DEBUG  :: FFmpeg binaries for Windows configured successfully!
Interface :: INFO :: Started recording
Interface :: INFO :: Stopped recording

I tried to pass output_params = {"-input_framerate":stream.framerate} as per your example, however it seems that isn't valid for ScreenGear:

I hit "AttributeError: 'ScreenGear' object has no attribute 'framerate'" as per https://github.com/abhiTronix/vidgear/issues/59.

I then tried with a hardcoded frame rate and saw similar behaviour to before (although my file is definetly compressed which is great!).

I recorded for 12s with output_params = {"-input_framerate": 30}, video was ~6s
I then recorded for 12s with output_params = {"-input_framerate": 15}, video was ~11s
I counted the frames and has 175 in 12s. Presumably I could get 1:1 playback if I set FPS to 175/12.

abhiTronix commented 2 years ago

I tried to pass output_params = {"-input_framerate":stream.framerate} as per your example, however it seems that isn't valid for ScreenGear:

Use desired one like 30 instead.

aza547 commented 2 years ago

I tried to pass output_params = {"-input_framerate":stream.framerate} as per your example, however it seems that isn't valid for ScreenGear:

Use desired one like 30 instead.

I think I tried this already (unless I'm minsunderstanding) as per my above comment:

I then tried with a hardcoded frame rate and saw similar behaviour to before (although my file is definetly compressed which is great!).

I recorded for 12s with output_params = {"-input_framerate": 30}, video was ~6s I then recorded for 12s with output_params = {"-input_framerate": 15}, video was ~11s I counted the frames and has 175 in 12s. Presumably I could get 1:1 playback if I set FPS to 175/12.

abhiTronix commented 2 years ago

I then tried with a hardcoded frame rate and saw similar behaviour to before (although my file is definetly compressed which is great!).

If -input_framerate attribute doesn't works for you, then define it in conjunction with another -r FFmpeg parameter as attribute:

# set output constant framerate to (say 30 fps)
output_params = {"-input_framerate":30, "-r":30}
# assign that to WriteGear
writer = WriteGear(output_filename="out.mp4", logging =True, **output_params)

abhiTronix commented 2 years ago

@alexanderkershaw Is python script taking too much memory, Check task manager?

aza547 commented 2 years ago

Looks like it peaks at 1.4GB memory usage during the 12s recording. Have plenty more free so doesn't look like it's bottlenecking there. I tried what you suggested but no luck - still the same result.

I seem to only have captured 180 frames during the 12s recording (less than 30 FPS - maybe this is the problem?)

I've attached a console log now with logging=True enabled log.txt

abhiTronix commented 2 years ago

I tried what you suggested but no luck - still the same result.

@alexanderkershaw Can you share the video here, just copy paste it into the comment.

aza547 commented 2 years ago

https://user-images.githubusercontent.com/32389325/143771447-b62a5933-4392-406a-a437-e8c8dfd5b3d9.mp4

abhiTronix commented 2 years ago

@alexanderkershaw I don't see any anomality, I think WriteGear is not the culprit but ScreenGear is capturing frames with delay.

abhiTronix commented 2 years ago

@alexanderkershaw See this:

abhiTronix commented 2 years ago

@alexanderkershaw Can you force backend="scrot" in ScreenGear as:

# open video stream with defined parameters and `mss` backend for extracting frames.
stream = ScreenGear(backend="scrot", logging=True).start()

aza547 commented 2 years ago

PS C:\checkouts\arena-recorder> python .\src\main.py
Interface :: DEBUG :: Started watching: WoWCombatLog-123.txt
Interface :: DEBUG :: Detected ARENA_MATCH_START event
Interface :: DEBUG :: It's 3v3 in Hook Point
14:19:09 ::   ScreenGear   ::  DEBUG  :: Enabling Threaded Queue Mode by default for ScreenGear!
Exception in thread Thread-1:
Traceback (most recent call last):
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\site-packages\easyprocess\__init__.py", line 168, in start
    self.popen = subprocess.Popen(
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\subprocess.py", line 1420, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\site-packages\vidgear\gears\screengear.py", line 150, in __init__
    self.__capture_object.grab(
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\site-packages\pyscreenshot\__init__.py", line 30, in grab
    return backend_grab(backend, bbox, childprocess)
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\site-packages\pyscreenshot\loader.py", line 156, in backend_grab
    return force(backend, bbox, childprocess)
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\site-packages\pyscreenshot\loader.py", line 150, in force
    im = obj.grab(bbox)
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\site-packages\pyscreenshot\plugins\scrot.py", line 20, in grab
    im = read_prog_img([PROGRAM, "--silent"])
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\site-packages\pyscreenshot\tempexport.py", line 27, in read_prog_img
    im = read_func_img(run_prog)
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\site-packages\pyscreenshot\tempexport.py", line 15, in read_func_img
    func(filename, bbox)
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\site-packages\pyscreenshot\tempexport.py", line 23, in run_prog
    p.call()
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\site-packages\easyprocess\__init__.py", line 141, in call
    self.start().wait(timeout=timeout)
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\site-packages\easyprocess\__init__.py", line 174, in start
    raise EasyProcessError(self, "start error")
easyprocess.EasyProcessError: start error <EasyProcess cmd_param=['scrot', '--silent', 'C:\\Users\\Alex\\AppData\\Local\\Temp\\pyscreenshotb260g1oh\\screenshot.png'] cmd=['scrot', '--silent', 'C:\\Users\\Alex\\AppData\\Local\\Temp\\pyscreenshotb260g1oh\\screenshot.png'] oserror=[WinError 2] The system cannot find the file specified return_code=None stdout="None" stderr="None" timeout_happened=False>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\threading.py", line 973, in _bootstrap_inner
    self.run()
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\threading.py", line 910, in run
    self._target(*self._args, **self._kwargs)
  File "C:\checkouts\arena-recorder\src\Watcher.py", line 48, in watch
    recorder = Recorder(
  File "C:\checkouts\arena-recorder\src\Recorder.py", line 14, in __init__
    self.stream = ScreenGear(backend="scrot", logging=True).start()
  File "C:\Users\Alex\AppData\Local\Programs\Python\Python39\lib\site-packages\vidgear\gears\screengear.py", line 190, in __init__
    raise SystemError(
SystemError: [ScreenGear:ERROR] :: Unable to grab any instance on this system, Are you running headless?

abhiTronix commented 2 years ago

@alexanderkershaw OK scrot isn't available on windows, try "mss" instead.

abhiTronix commented 2 years ago

@alexanderkershaw Also, try "pil" backend.

aza547 commented 2 years ago

With mss I captured 77 frames over 12s and final video was 2s long
With pil I captured 177 frames over 12s and final video was 5s long

aza547 commented 2 years ago

I added some time.time() calls like this

t1 = time.time()
frame = self.stream.read()
t2 = time.time()
print("frame read time:", t2-t1)

and saw:

frame read time: 0.030254602432250977
frame write time: 0.027775049209594727
frame read time: 0.03469228744506836
frame write time: 0.027278423309326172
frame read time: 0.029759883880615234
frame write time: 0.027751922607421875
frame read time: 0.040178775787353516
frame write time: 0.03025650978088379
frame read time: 0.028246164321899414

Interesting that the frame read time has gone up a lot since I initially raised the issue and the write time has gone down slightly...

abhiTronix commented 2 years ago

With mss I captured 77 frames over 12s and final video was 2s long With pil I captured 177 frames over 12s and final video was 5s long

@alexanderkershaw Ok now add this flag to existing flags in WriteGear:

output_params = {"-disable_force_termination": True} # disable the default forced-termination behaviour
# assign that to WriteGear
writer = WriteGear(output_filename="out.mp4", logging =True, **output_params)

and then retry these tests

abhiTronix commented 2 years ago

Interesting that the frame read time has gone up a lot since I initially raised the issue and the write time has gone down slightly...

That's because vidgear is optimize to perform better under more load. So when you're using OpenCV backend, the system IO was under stress, and that's where vidgear shines. Now that writing is fast, the result is barely noticeable or bad than previous.

aza547 commented 2 years ago

Results:

default: 174 frames in 12s
mss: 73 frames in 12s
pil: 178 frames in 12s

abhiTronix commented 2 years ago

Results:

default: 174 frames in 12s mss: 73 frames in 12s pil: 178 frames in 12s

So no effect. :/

aza547 commented 2 years ago

Seems that way yes :(

abhiTronix commented 2 years ago

Can you comment out writer and instead put cv2.imshow("Output Frame", frame) to confirm if frames are not skipping there.

Put this:

    # Show output window
    cv2.imshow("Output Frame", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

aza547 commented 2 years ago

Up to 277 frames there (but I suppose that's not a suprise as removing the write has removed approx half the time consuming operations in the loop?)

abhiTronix commented 2 years ago

Up to 277 frames there (but I suppose that's not a suprise as removing the write has removed approx half the time consuming operations in the loop?)

That's unusual behaviour. But can you see skipping there?

aza547 commented 2 years ago

Nope no skipping in the window that is opened there.

abhiTronix commented 2 years ago

Nope no skipping in the window that is opened there.

Also with pil or pil backend is skipping frames?

aza547 commented 2 years ago

Stepping back a bit I think I have two issues:

Not managing to read/write frames fast enough.
The video saved still needs a hardcoded framerate in code.

I think that even if I solve issue 1 above I'm still going to have issue 2, is that correct? Your original reply to my question stated "Use constant framerate in compression mode." - I'm doing that now and the video playback is still not quite the right speed.

I think I was expecting this 2 to be solved by swapping to compression mode - and my performance issue would just cause laggy video, but doesn't seem to be the case.

aza547 commented 2 years ago

Also with pil or pil backend is skipping frames?

Same with pil, no skipping

abhiTronix commented 2 years ago

Your original reply to my question stated "Use constant framerate in compression mode." - I'm doing that now and the video playback is still not quite the right speed.

I'm not seeing any anomality in logs and none in output video you shared, I'm still unsure why this is happening.

Results: default: 174 frames in 12s mss: 73 frames in 12s pil: 178 frames in 12s

@alexanderkershaw With pil+WriteGear, skipping is there right?

abhiTronix commented 2 years ago

Try reducing frame size:

Add this line to import:

from vidgear.gears.helper import reducer

Add this line before writing frame:

frame = reducer(frame, percentage=25)  # reduce frame by 25%

aza547 commented 2 years ago

@alexanderkershaw With pil+WriteGear, skipping is there right?

don't think there is skipping here (when you say skipping - you mean choppy playback? - sorry if that's obvious - have no background in video!)

https://user-images.githubusercontent.com/32389325/143773388-0ba55a63-0559-4e80-a6ca-c89a774714db.mp4

aza547 commented 2 years ago

Try reducing frame size:

Add this line to import:
from vidgear.gears.helper import reducer
Add this line before writing frame:
frame = reducer(frame, percentage=25)  # reduce frame by 25%

Seemingly no effect from this. Although this video file size has got larger.

PS C:\checkouts\arena-recorder> python .\src\main.py
Interface :: DEBUG :: Started watching: WoWCombatLog-123.txt
Interface :: DEBUG :: Detected ARENA_MATCH_START event
Interface :: DEBUG :: It's 3v3 in Hook Point
14:57:48 ::   ScreenGear   ::  DEBUG  :: Enabling Threaded Queue Mode by default for ScreenGear!
14:57:48 ::   WriteGear    ::  DEBUG  :: Compression Mode is enabled therefore checking for valid FFmpeg executable.
14:57:48 ::   WriteGear    ::  DEBUG  :: Output Parameters: {'-input_framerate': 30, '-r': 30, '-disable_force_termination': True}
14:57:48 ::     Helper     ::  DEBUG  :: FFmpeg Windows Download Path: C:\Users\Alex\AppData\Local\Temp
14:57:48 ::     Helper     ::  DEBUG  :: Final FFmpeg Path: C:\Users\Alex\AppData\Local\Temp\ffmpeg-static-win64-gpl/bin/ffmpeg.exe
14:57:48 ::     Helper     ::  DEBUG  :: FFmpeg validity Test Passed!
14:57:48 ::     Helper     ::  DEBUG  :: Found valid FFmpeg Version: `b'N-104643-g7e9e2cf93b-20211125'` installed on this system
14:57:48 ::   WriteGear    ::  DEBUG  :: Found valid FFmpeg executable: `C:\Users\Alex\AppData\Local\Temp\ffmpeg-static-win64-gpl/bin/ffmpeg.exe`.
14:57:48 ::   WriteGear    ::  DEBUG  :: Compression Mode with FFmpeg backend is configured properly.
Interface :: INFO :: Started recording
14:57:48 ::   WriteGear    ::  DEBUG  :: InputFrame => Height:810 Width:1440 Channels:3
14:57:48 ::   WriteGear    ::  DEBUG  :: Setting Input framerate: 30.0
14:57:48 ::   WriteGear    ::  DEBUG  :: Executing FFmpeg command: `C:\Users\Alex\AppData\Local\Temp\ffmpeg-static-win64-gpl/bin/ffmpeg.exe -y -f rawvideo -vcodec rawvideo -s 1440x810 -pix_fmt bgr24 -framerate 30.0 -i - -r 30 -vcodec libx264 -crf 18 -preset fast F:\arena-recorder-movies\3v3-Hook Point-14.57.48.mp4`
ffmpeg version N-104643-g7e9e2cf93b-20211125 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10-win32 (GCC) 20210408
  configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- --arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads --enable-iconv --enable-libxml2 --enable-zlib --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig --enable-libvorbis --enable-opencl --enable-libvmaf --enable-vulkan --disable-libxcb --disable-xlib --enable-amf --enable-libaom --enable-avisynth --enable-libdav1d --enable-libdavs2 --disable-libfdk-aac --enable-ffnvcodec --enable-cuda-llvm --enable-frei0r --enable-libglslang --enable-libgme --enable-libass --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvpx --enable-libwebp --enable-lv2 --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --disable-libdrm --disable-vaapi --enable-libvidstab --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-ldexeflags= --extra-libs=-lgomp --extra-version=20211125
  libavutil      57.  9.101 / 57.  9.101
  libavcodec     59. 13.100 / 59. 13.100
  libavformat    59.  9.101 / 59.  9.101
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8. 17.100 /  8. 17.100
  libswscale      6.  1.100 /  6.  1.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Input #0, rawvideo, from 'pipe:':
  Duration: N/A, start: 0.000000, bitrate: 839808 kb/s
  Stream #0:0: Video: rawvideo (BGR[24] / 0x18524742), bgr24, 1440x810, 839808 kb/s, 30 tbr, 30 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
[libx264 @ 000001b0489bc400] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 000001b0489bc400] profile High 4:4:4 Predictive, level 3.2, 4:4:4, 8-bit
[libx264 @ 000001b0489bc400] 264 - core 164 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=2 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=6 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=30 rc=crf mbtree=1 crf=18.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'F:\arena-recorder-movies\3v3-Hook Point-14.57.48.mp4':
  Metadata:
    encoder         : Lavf59.9.101
  Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv444p(tv, progressive), 1440x810, q=2-31, 30 fps, 15360 tbn
    Metadata:
      encoder         : Lavc59.13.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
Interface :: DEBUG :: Detected ARENA_MATCH_END event:00.00 bitrate=N/A speed=   0x
Interface :: INFO :: Stopped recording04kB time=00:00:03.76 bitrate=5010.9kbits/s speed=0.331x      x
iL 172
14:58:00 ::   ScreenGear   ::  DEBUG  :: Terminating ScreenGear Processes.
14:58:00 ::   WriteGear    ::  DEBUG  :: Terminating WriteGear Processes.
frame=  172 fps= 14 q=-1.0 Lsize=    2683kB time=00:00:05.63 bitrate=3900.9kbits/s speed=0.465x
video:2680kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.106963%
[libx264 @ 000001b0489bc400] frame I:1     Avg QP:17.14  size:207811
[libx264 @ 000001b0489bc400] frame P:47    Avg QP:18.35  size: 39070
[libx264 @ 000001b0489bc400] frame B:124   Avg QP:23.99  size:  5638
[libx264 @ 000001b0489bc400] consecutive B-frames:  1.7%  4.7%  5.2% 88.4%
[libx264 @ 000001b0489bc400] mb I  I16..4: 49.3% 19.7% 31.0%
[libx264 @ 000001b0489bc400] mb P  I16..4:  3.5%  2.0%  3.9%  P16..4:  5.1%  1.0%  0.7%  0.0%  0.0%    skip:83.8%
[libx264 @ 000001b0489bc400] mb B  I16..4:  0.4%  0.0%  0.3%  B16..8:  1.8%  0.4%  0.3%  direct: 0.7%  skip:95.9%  L0:33.8% L1:55.4% BI:10.8%
[libx264 @ 000001b0489bc400] 8x8 transform intra:17.9% inter:37.0%
[libx264 @ 000001b0489bc400] coded y,u,v intra: 41.1% 35.6% 34.5% inter: 1.4% 1.2% 1.2%
[libx264 @ 000001b0489bc400] i16 v,h,dc,p: 58% 37%  5%  1%
[libx264 @ 000001b0489bc400] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 55% 19% 19%  1%  1%  1%  1%  1%  2%
[libx264 @ 000001b0489bc400] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 28% 28% 16%  3%  3%  5%  6%  4%  6%
[libx264 @ 000001b0489bc400] Weighted P-Frames: Y:2.1% UV:2.1%
[libx264 @ 000001b0489bc400] ref P L0: 64.0% 36.0%
[libx264 @ 000001b0489bc400] ref B L0: 87.2% 12.8%
[libx264 @ 000001b0489bc400] ref B L1: 96.5%  3.5%
[libx264 @ 000001b0489bc400] kb/s:3827.82

abhiTronix commented 2 years ago

don't think there is skipping here (when you say skipping - you mean choppy playback? - sorry if that's obvious - have no background in video!)

But you said,

I think that even if I solve issue 1 above I'm still going to have issue 2, is that correct? Your original reply to my question stated "Use constant framerate in compression mode." - I'm doing that now and the video playback is still not quite the right speed.

Isn't video is faster than require?

aza547 commented 2 years ago

Isn't video is faster than require?

Yes - as in the stopwatch videos the 6s of video was 12s of real time - I think that's always happened in every repro attempt, but the we've had some variance in the performance in regard to how many frames are being captured so the ratio has changed with this.

aza547 commented 2 years ago

https://stackoverflow.com/questions/46545374/how-to-acheive-1x-writing-speed-for-4k-encoding-in-ffmpeg/46684054 mentions the -re flag, i don't see that anywhere in the logs.

from the ffmpeg docs https://ffmpeg.org/ffmpeg.html

-re (input) Read input at native frame rate. This is equivalent to setting -readrate 1.

Should this argument be present?

abhiTronix commented 2 years ago

I ran some tests, and @alexanderkershaw I think it is impossible to achieve exact framerate as real-time, as different frames available for processing at different times, and it makes impossible for FFmpeg to predict perfect framerate. I check through FFmpeg docs and I failed to find any solution to this problem, you always going to have some minor anomaly in output framerate. Also, python isn't a best choice for exact real-time application, you should look into C++ solutions.

aza547 commented 2 years ago

OK - I think a minor anomaly would be OK but currently my videos are varying by up to 50% from the real time.

Anyway thanks for all your efforts - I'll look into other solutions/C++.

abhiTronix commented 2 years ago

OK - I think a minor anomaly would be OK but currently my videos are varying by up to 50% from the real time.

@alexanderkershaw I tried and found lower framerate like 10 works better.

abhiTronix commented 2 years ago

https://stackoverflow.com/questions/46545374/how-to-acheive-1x-writing-speed-for-4k-encoding-in-ffmpeg/46684054 mentions the -re flag, i don't see that anywhere in the logs.

from the ffmpeg docs https://ffmpeg.org/ffmpeg.html

@alexanderkershaw Sorry I didn't see your comment. It is removed in earlier version. I'll test it please wait.

abhiTronix commented 2 years ago

@alexanderkershaw Nah no effect.

abhiTronix commented 2 years ago

What is the correct way to ensure that recording FPS and video FPS are the same? I want real-time video playback in an app I'm writing so videos that don't match up 1:1 in time spent recording / time spent playing is a problem for me.

@alexanderkershaw Ok I've solved this problem, actually you need a different approach to solve this. You need to completely eliminate ScreenGear and do everything in WriteGear. Here's the code generate time accurate Video with WriteGear:

# import required libraries
from vidgear.gears import WriteGear

# Define writer with defined parameters
writer = WriteGear(output_filename="Output.mp4", logging=True) # note: Output.mp4 is dummy name 

# format FFmpeg command
ffmpeg_command = [
    "-y", # `-y` parameter is to overwrite outputfile if exists
    "-f",
    "gdigrab",
    "-framerate",
    "30", # increase this if want more accurate results.
    "-i",
    "desktop",
    "output.mkv", # this is output file name
]  

# execute FFmpeg command
writer.execute_ffmpeg_cmd(ffmpeg_command)

# close
writer.close()

Kindly test it and confirm if this works.

rolandqi commented 2 years ago

Hi @abhiTronix I've tried this code and seems it worked well. But by doing this, the current thread waits for the call execute_ffmpeg_cmd to finish. Is there any way to stop the call of execute_ffmpeg_cmd and safely save the output vedio file from another python thread?

abhiTronix commented 2 years ago

@rolandqi No not yet, wait for #148 to resolve.

abhiTronix commented 1 year ago

Anyone who want to fast frames from Desktop in 2023, try this instead: https://abhitronix.github.io/deffcode/latest/recipes/advanced/decode-live-feed-devices/#capturing-and-previewing-frames-from-your-desktop

abhiTronix / vidgear