search

MaartenBaert / ssr

SimpleScreenRecorder, a screen recorder for Linux
http://www.maartenbaert.be/simplescreenrecorder/
GNU General Public License v3.0
2.53k stars 286 forks source link

SSR OpenGL capture gets confused by the presence of multiple streams (WINE) #597

Open haasn opened 6 years ago

haasn commented 6 years ago

Reproduction:

$ ssr-glinject wine 'Diablo III.exe' -launch
ssr-glinject: LD_PRELOAD = :libssr-glinject.so
ssr-glinject: command = wine Diablo III.exe -launch
[SSR-GLInject] Library loaded (64-bit).
[SSR-GLInject] Library loaded (32-bit).
[SSR-GLInject] Library loaded (32-bit).
[SSR-GLInject] Library loaded (64-bit).
[SSR-GLInject] Library loaded (64-bit).
fixme:winediag:start_process Wine Staging 2.21 is a testing version containing experimental patches.
fixme:winediag:start_process Please mention your exact version when filing bug reports on winehq.org.
[SSR-GLInject] Library loaded (64-bit).
[SSR-GLInject] Library loaded (64-bit).
[SSR-GLInject] Library loaded (64-bit).
[SSR-GLInject] Library loaded (32-bit).
...
[SSR-GLInject] Hooked: dlsym(glXGetProcAddressARB).
[SSR-GLInject] Hooked: glXGetProcAddressARB(glXSwapBuffers).
[SSR-GLInject] Hooked: glXGetProcAddressARB(glXCreateWindow).
[SSR-GLInject] Hooked: glXGetProcAddressARB(glXDestroyWindow).
err:wininet:open_http_connection create_netconn failed: 12029
[SSR-GLInject] [GLXFrameGrabber 1] Created GLX frame grabber.
[SSR-GLInject] [/dev/shm/ssr-channel-nand/video-515044535167-5650-glx0001-wine-preloader] Created video stream.
fixme:heap:GetPhysicallyInstalledSystemMemory stub: 0x33f938
[SSR-GLInject] [GLXFrameGrabber 2] Created GLX frame grabber.
[SSR-GLInject] [/dev/shm/ssr-channel-nand/video-515044568962-5650-glx0002-wine-preloader] Created video stream.
...
[SSR-GLInject] Hooked: dlsym(glXGetProcAddressARB).
[SSR-GLInject] Hooked: glXGetProcAddressARB(glXSwapBuffers).
[SSR-GLInject] Hooked: glXGetProcAddressARB(glXCreateWindow).
[SSR-GLInject] Hooked: glXGetProcAddressARB(glXDestroyWindow).
[SSR-GLInject] [GLXFrameGrabber 1] Created GLX frame grabber.
[SSR-GLInject] [/dev/shm/ssr-channel-nand/video-515052960995-5590-glx0001-wine-preloader] Created video stream.
...
[SSR-GLInject] [GLXFrameGrabber 2] Created GLX frame grabber.
[SSR-GLInject] [/dev/shm/ssr-channel-nand/video-515053057247-5590-glx0002-wine-preloader] Created video stream.
fixme:d3d:wined3d_query_create Unhandled query type 0x11.
fixme:d3d:wined3d_query_create Unhandled query type 0x12.
fixme:d3d:wined3d_query_create Unhandled query type 0xe.
fixme:d3d:wined3d_query_create Unhandled query type 0xd.
fixme:d3d:wined3d_query_create Unhandled query type 0x6.
fixme:d3d:wined3d_query_create Unhandled query type 0xf.
fixme:d3d:wined3d_query_create Unhandled query type 0x5.
[SSR-GLInject] [GLXFrameGrabber 3] Created GLX frame grabber.
[SSR-GLInject] [/dev/shm/ssr-channel-nand/video-515053105097-5590-glx0003-wine-preloader] Created video stream.
[SSR-GLInject] [GLXFrameGrabber 4] Created GLX frame grabber.
[SSR-GLInject] [/dev/shm/ssr-channel-nand/video-515053123293-5590-glx0004-wine-preloader] Created video stream.
...
[SSR-GLInject] OpenGL version = 4.6 (4.6.0 NVIDIA 387.34).
[SSR-GLInject] [/dev/shm/ssr-channel-nand/video-515053057247-5590-glx0002-wine-preloader] frame size = 4096x2128.
[SSR-GLInject] [/dev/shm/ssr-channel-nand/video-515053057247-5590-glx0002-wine-preloader] frame size = 2046x2126.
[SSR-GLInject] [/dev/shm/ssr-channel-nand/video-515053057247-5590-glx0002-wine-preloader] frame size = 4096x2128.
[SSR-GLInject] [/dev/shm/ssr-channel-nand/video-515053057247-5590-glx0002-wine-preloader] frame size = 2046x2126.
[SSR-GLInject] [/dev/shm/ssr-channel-nand/video-515053057247-5590-glx0002-wine-preloader] frame size = 4096x2128.
fixme:winhttp:get_system_proxy_autoconfig_url no support on this platform
fixme:winhttp:WinHttpDetectAutoProxyConfigUrl discovery via DHCP not supported
[SSR-GLInject] [/dev/shm/ssr-channel-nand/video-515053057247-5590-glx0002-wine-preloader] frame size = 2046x2126.
[SSR-GLInject] [/dev/shm/ssr-channel-nand/video-515053057247-5590-glx0002-wine-preloader] frame size = 4096x2128.
[SSR-GLInject] [/dev/shm/ssr-channel-nand/video-515053057247-5590-glx0002-wine-preloader] frame size = 2046x2126.

As you can see, it opens up a whole bunch of video streams, but onl the glx0002 is the correct one (it's the only one that has a valid frame size).

The SSR program gets confused and can't handle this:

[PageRecord::StartPage] Starting page ...
[SSRVideoStreamWatcher::Init] Added pre-existing stream video-515053123293-5590-glx0004-wine-preloader.
[SSRVideoStreamWatcher::Init] Added pre-existing stream video-515053105097-5590-glx0003-wine-preloader.
[SSRVideoStreamWatcher::Init] Added pre-existing stream video-515053057247-5590-glx0002-wine-preloader.
[SSRVideoStreamWatcher::Init] Added pre-existing stream video-515052960995-5590-glx0001-wine-preloader.
[SSRVideoStreamWatcher::Init] Added pre-existing stream video-515044568962-5650-glx0002-wine-preloader.
[SSRVideoStreamWatcher::Init] Added pre-existing stream video-515044535167-5650-glx0001-wine-preloader.
[PageRecord::StartPage] Started page.
[GLInjectInput::InputThread] Input thread started.
[SSRVideoStreamReader::Init] Created video stream reader.
[PageRecord::StartOutput] Starting output ...
[PageRecord::StartOutput] Error: Could not get the size of the OpenGL application. Either the application wasn't started correctly, or the application hasn't created an OpenGL window yet. If you want to start recording before starting the application, you have to enable scaling and enter the video size manually.
[PageRecord::StartOutput] Error: Something went wrong during initialization.

As a crude work-around, I can use a different channel and only symlink the video stream I want:

$ ll ssr-channel-nand
total 140M
-rw------- 1 nand nand 4.1k Dec 13 21:28 video-515044535167-5650-glx0001-wine-preloader
-rw------- 1 nand nand 4.1k Dec 13 21:28 video-515044568962-5650-glx0002-wine-preloader
-rw------- 1 nand nand 4.1k Dec 13 21:28 video-515052960995-5590-glx0001-wine-preloader
-rw------- 1 nand nand 4.1k Dec 13 21:33 video-515053057247-5590-glx0002-wine-preloader
-rw------- 1 nand nand 4.1k Dec 13 21:28 video-515053105097-5590-glx0003-wine-preloader
-rw------- 1 nand nand 4.1k Dec 13 21:37 video-515053123293-5590-glx0004-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe0-515044535167-5650-glx0001-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe0-515044568962-5650-glx0002-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe0-515052960995-5590-glx0001-wine-preloader
-rw------- 1 nand nand  44M Dec 13 21:33 videoframe0-515053057247-5590-glx0002-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe0-515053105097-5590-glx0003-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe0-515053123293-5590-glx0004-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe1-515044535167-5650-glx0001-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe1-515044568962-5650-glx0002-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe1-515052960995-5590-glx0001-wine-preloader
-rw------- 1 nand nand  44M Dec 13 21:33 videoframe1-515053057247-5590-glx0002-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe1-515053105097-5590-glx0003-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe1-515053123293-5590-glx0004-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe2-515044535167-5650-glx0001-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe2-515044568962-5650-glx0002-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe2-515052960995-5590-glx0001-wine-preloader
-rw------- 1 nand nand  44M Dec 13 21:33 videoframe2-515053057247-5590-glx0002-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe2-515053105097-5590-glx0003-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe2-515053123293-5590-glx0004-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe3-515044535167-5650-glx0001-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe3-515044568962-5650-glx0002-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe3-515052960995-5590-glx0001-wine-preloader
-rw------- 1 nand nand  44M Dec 13 21:33 videoframe3-515053057247-5590-glx0002-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe3-515053105097-5590-glx0003-wine-preloader
-rw------- 1 nand nand    0 Dec 13 21:28 videoframe3-515053123293-5590-glx0004-wine-preloader

$ ll ll ssr-channel-test
total 0
lrwxrwxrwx 1 nand nand 66 Dec 13 21:33 video-515053057247-5590-glx0002-wine-preloader -> ../ssr-channel-nand/video-515053057247-5590-glx0002-wine-preloader
lrwxrwxrwx 1 nand nand 72 Dec 13 21:33 videoframe0-515053057247-5590-glx0002-wine-preloader -> ../ssr-channel-nand/videoframe0-515053057247-5590-glx0002-wine-preloader
lrwxrwxrwx 1 nand nand 72 Dec 13 21:33 videoframe1-515053057247-5590-glx0002-wine-preloader -> ../ssr-channel-nand/videoframe1-515053057247-5590-glx0002-wine-preloader
lrwxrwxrwx 1 nand nand 72 Dec 13 21:33 videoframe2-515053057247-5590-glx0002-wine-preloader -> ../ssr-channel-nand/videoframe2-515053057247-5590-glx0002-wine-preloader
lrwxrwxrwx 1 nand nand 72 Dec 13 21:33 videoframe3-515053057247-5590-glx0002-wine-preloader -> ../ssr-channel-nand/videoframe3-515053057247-5590-glx0002-wine-preloader

Now, if I set the channel name to channel-test in SSR, it works fine:

[PageRecord::StartPage] Starting page ...
[SSRVideoStreamWatcher::Init] Added pre-existing stream video-515053057247-5590-glx0002-wine-preloader.
[PageRecord::StartPage] Started page.
[GLInjectInput::InputThread] Input thread started.
[SSRVideoStreamReader::Init] Created video stream reader.
[PageRecord::StartOutput] Starting output ...
[Muxer::Init] Using format matroska (Matroska).
[Muxer::AddStream] Using codec libx264 (libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10).
[VideoEncoder::PrepareStream] Using pixel format nv12.
[libx264 @ 0x5584c2e42560] Application has requested 32 threads. Using a thread count greater than 16 is not recommended.
[libx264 @ 0x5584c2e42560] using SAR=1/1
[libx264 @ 0x5584c2e42560] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x5584c2e42560] profile High, level 5.2
[libx264 @ 0x5584c2e42560] 264 - core 152 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x3 me=dia subme=1 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=32 lookahead_threads=5 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 pb_ratio=1.30 aq=1:1.00
[Muxer::MuxerThread] Muxer thread started.
[PageRecord::StartOutput] Started output.
[BaseEncoder::EncoderThread] Encoder thread started.
[PageRecord::StartInput] Starting input ...
[PageRecord::StartInput] Started input.
[Synchronizer::SynchronizerThread] Synchronizer thread started.
[PageRecord::StopOutput] Stopping output ...
[PageRecord::StopOutput] Stopped output.
[PageRecord::StopInput] Stopping input ...
[PageRecord::StopInput] Stopped input.
[PageRecord::StopPage] Stopping page ...
[Synchronizer::~Synchronizer] Stopping synchronizer thread ...
[Synchronizer::SynchronizerThread] Synchronizer thread stopped.
[Muxer::Finish] Finishing encoders ...
[BaseEncoder::EncoderThread] Flushing encoder ...
[BaseEncoder::EncoderThread] Encoder thread stopped.
[Muxer::MuxerThread] Muxer thread stopped.
[Muxer::~Muxer] Stopping encoders ...
[Muxer::~Muxer] Waiting for muxer thread to stop ...
[BaseEncoder::~BaseEncoder] Stopping encoder thread ...
[libx264 @ 0x5584c2e42560] frame I:1     Avg QP:15.35  size:297897
[libx264 @ 0x5584c2e42560] frame P:58    Avg QP:14.47  size: 79606
[libx264 @ 0x5584c2e42560] frame B:143   Avg QP:16.71  size: 21480
[libx264 @ 0x5584c2e42560] consecutive B-frames:  5.4%  0.0%  1.5% 93.1%
[libx264 @ 0x5584c2e42560] mb I  I16..4: 11.2% 55.7% 33.1%
[libx264 @ 0x5584c2e42560] mb P  I16..4:  2.4%  6.1%  1.3%  P16..4: 33.9%  0.0%  0.0%  0.0%  0.0%    skip:56.3%
[libx264 @ 0x5584c2e42560] mb B  I16..4:  0.4%  0.4%  0.0%  B16..8: 10.6%  0.0%  0.0%  direct:10.5%  skip:78.0%  L0:35.5% L1:53.8% BI:10.7%
[libx264 @ 0x5584c2e42560] 8x8 transform intra:59.2% inter:31.4%
[libx264 @ 0x5584c2e42560] coded y,uvDC,uvAC intra: 60.7% 62.4% 33.7% inter: 7.5% 12.0% 1.4%
[libx264 @ 0x5584c2e42560] i16 v,h,dc,p: 45% 16% 32%  8%
[libx264 @ 0x5584c2e42560] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 10% 26%  8%  8%  9%  5%  8%  6%
[libx264 @ 0x5584c2e42560] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 12% 14%  9%  7% 10%  4%  9%  5%
[libx264 @ 0x5584c2e42560] i8c dc,h,v,p: 45% 17% 29%  9%
[libx264 @ 0x5584c2e42560] Weighted P-Frames: Y:1.7% UV:0.0%
[libx264 @ 0x5584c2e42560] kb/s:6678.73
[GLInjectInput::~GLInjectInput] Stopping input thread ...
[GLInjectInput::InputThread] Input thread stopped.
[SSRVideoStreamReader::Init] Destroyed video stream reader.
[PageRecord::StopPage] Stopped page.
MaartenBaert commented 6 years ago

The game must be opening multiple Direct3D contexts (possibly for multi-threaded rendering or something like that), which WINE is translating into multiple OpenGL contexts. The weird thing is that WINE is apparently creating actual windows for all of these contexts which clearly aren't actually used for displaying anything.

Unfortunately the current streaming protocol is very primitive, there is no easy way to check from SSR which of these streams is actually active without opening all streams at the same time, which is problematic. I was actually planning to change the protocol to a more advanced socket-based one, which would make it easier to solve problems like this. But for now there isn't much I can do.

haasn commented 6 years ago

Unfortunately the current streaming protocol is very primitive, there is no easy way to check from SSR which of these streams is actually active without opening all streams at the same time, which is problematic. I was actually planning to change the protocol to a more advanced socket-based one, which would make it easier to solve problems like this. But for now there isn't much I can do.

Since the “frame size = XYZ” message isn't printed for most of them, can't you just delay creating the shm socket until you actually know the frame size?

The game must be opening multiple Direct3D contexts (possibly for multi-threaded rendering or something like that)

Yeah dunno. It almost certainly isn't multi-threaded rendering because the game is ancient, poorly written and only runs on a single thread anyway. It's also 100% cpu bound. I do use CSMT, but I also tried with CSMT disabled and the result was the same.