search

wang-bin / QtAV

A cross-platform multimedia framework based on Qt and FFmpeg. 基于Qt和FFmpeg的跨平台高性能音视频播放框架. Recommand to use new sdk https://github.com/wang-bin/mdk-sdk
http://qtav.org
3.98k stars 1.5k forks source link

GLWidgetRenderer2 renders black screen instead of video. #500

Open ThatAIGeek opened 9 years ago

ThatAIGeek commented 9 years ago

Hi! I've tried to use QtAV for my desktop streaming application and I've run into issue that with WidgetRenderer I'm getting really poor productivity(which is understandable) while GLWidgetRenderer2 doesn't render anything. I start desktop streaming using ffmpeg and dshow screen-capture-recorder device: ffmpeg.exe -fflags nobuffer -f dshow -i video=\"screen-capture-recorder\" -c:v mpeg4 -tune zerolatency -q:v 5 -f mpegts udp://127.0.0.1:1234??pkt_size=188?buffer_size=65535 to test if stream is running you can use ffplay: ffplay.exe -fflags nobuffer udp://127.0.0.1:1234??pkt_size=188?buffer_size=65535 I'm on windows, gcc5.1.0 sjlj, Qt 4.8.6, QtAV latest from repo

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/26420336-glwidgetrenderer2-renders-black-screen-instead-of-video?utm_campaign=plugin&utm_content=tracker%2F307703&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F307703&utm_medium=issues&utm_source=github).
wang-bin commented 9 years ago

I tested my camera and it works fine. Show me your log. (set environment var QTAV_LOG=all in qtcreator)

ThatAIGeek commented 9 years ago 
"QtAV 1.8.0(Sep  3 2015, 17:18:06)
A media playing library base on Qt and FFmpeg.
Distributed under the terms of LGPLv2.1 or later.
Shanghai University->S3 Graphics->Deepin, Shanghai, ChinaCopyright (C) 2012-2015 Wang Bin (aka. Lucas Wang) wbsecg1@gmail.com
Donate: http://www.qtav.org/donate.html
Source: https://github.com/wang-bin/QtAV
Home page: http://www.qtav.org" 
Build with Qt-4.8.6
Build with avutil-54.31.100
Build with avcodec-56.58.100
Build with avformat-56.40.101
Build with avfilter-5.37.100
Build with avdevice-56.4.100
Build with swresample-1.2.101
Build with swscale-3.1.101
Registered audio backends:  ("null", "DirectSound", "XAudio2") 
"XAudio2_9" 
"Cannot load library XAudio2_9: The specified module could not be found." 
"XAudio2_8" 
Build with XAudio2 from DXSDK
Try symbol 'XAudio2Create' from WinSDK dll
xaudio2: 0x10f20560
OpenGL version: 4.3  hasGLSL: 1
GL_VERSION: 4.3.0 - Build 10.18.14.4170
GL_VENDOR: Intel
GL_RENDERER: Intel(R) HD Graphics 4600
GL_SHADING_LANGUAGE_VERSION: 4.30 - Build 10.18.14.4170
Loading  QVariant(QString, "udp://127.0.0.1:25055")  ... 
all closed and reseted
dict: fflags=>nobuffer
dict: tune=>zerolatency,fastdecode
avformat_open_input: d->format_ctx:'0xd4b9e40', url:'udp://127.0.0.1:25055'...
avformat_open_input: url:'udp://127.0.0.1:25055' ret:0
avfmtctx.flag: 64
AVFMT_NOTIMESTAMPS: 0, AVFMT_TS_DISCONT: 0, AVFMT_NO_BYTE_SEEK:0
invalid index 0 (valid is 0~0) for stream type 2.
notify_interval: 500
stream: 0, duration=-9223372036854775808 (-102481911520608624 ms), time_base=0.000011
codec: mpeg4(MPEG-4 part 2)
invalid index 0 (valid is 0~0) for stream type 0.
**********trying video decoder: FFmpeg...
**************Video decoder found:0xd4dc2d8
setCapture 0xd6003c0
stream: 0, duration=-9223372036854775808 (-102481911520608620 ms), time_base=0.000011
codec: mpeg4(MPEG-4 part 2)
Starting video thread...
get av queue a/v thread = 0x0 0xf2b6cf8
AVClock started!!!!!!!!
Invalid packet! flush video codec context!!!!!!!!!! video packet queue size: 14
waiting for key frame. queue size: 13. pkt.size: 5051
waiting for key frame. queue size: 12. pkt.size: 4295
waiting for key frame. queue size: 11. pkt.size: 6541
waiting for key frame. queue size: 10. pkt.size: 5101
waiting for key frame. queue size: 9. pkt.size: 4013
waiting for key frame. queue size: 8. pkt.size: 4400
waiting for key frame. queue size: 7. pkt.size: 4514
waiting for key frame. queue size: 6. pkt.size: 3067
waiting for key frame. queue size: 5. pkt.size: 2687
waiting for key frame. queue size: 4. pkt.size: 2665
waiting for key frame. queue size: 3. pkt.size: 2858
no frame could be decompressed: Error number 111467 occurred 0/111467
invalid video frame from decoder. undecoded data size: 0
no frame could be decompressed: Error number 3593 occurred 0/3593
invalid video frame from decoder. undecoded data size: 0
no frame could be decompressed: Error number 2257 occurred 0/2257
invalid video frame from decoder. undecoded data size: 0
no frame could be decompressed: Error number 50651 occurred 0/50651
invalid video frame from decoder. undecoded data size: 0
no frame could be decompressed: Error number 18882 occurred 0/18882
invalid video frame from decoder. undecoded data size: 0
no frame could be decompressed: Error number 20726 occurred 0/20726
invalid video frame from decoder. undecoded data size: 0
no frame could be decompressed: Error number 18553 occurred 0/18553
invalid video frame from decoder. undecoded data size: 0
no frame could be decompressed: Error number 17253 occurred 0/17253
invalid video frame from decoder. undecoded data size: 0
setInSize => calculating aspect ratio from converted input data(1.777778)
QtAV::VideoFormat(pixelFormat: 15 "yuv420p" has alpha: false, channels: 3, planes: 3, bitsPerPixel: 12) 
pixel format changed:  => yuv420p 15
"[ShaderManager] cache a new shader material type(19): gl material planar: 1, has alpha: 0, big endian: 0, 2d texture: 1, 8bit channel: 1" 
bind attribute: a_Position => 0
bind attribute: a_TexCoords0 => 1
glGetUniformLocation("u_Texture0") = 2
glGetUniformLocation("u_Texture1") = 3
glGetUniformLocation("u_Texture2") = 4
glGetUniformLocation("u_MVP_matrix") = 0
glGetUniformLocation("u_colorMatrix") = 1
glGetUniformLocation("u_opacity") = 5
plane linesize 0: padded = 1920, effective = 1920
plane width 1920: effective = 1920
planeHeight 0 = 1080
plane linesize 1: padded = 960, effective = 960
plane width 960: effective = 960
planeHeight 1 = 540
plane linesize 2: padded = 960, effective = 960
plane width 960: effective = 960
planeHeight 2 = 540
effective_tex_width_ratio=1.000000
///////////bpp 2
texture width: 1920 - 1920 = pad: 0. bpp(gl): 1
texture width: 960 - 960 = pad: 0. bpp(gl): 1
texture width: 960 - 960 = pad: 0. bpp(gl): 1
creating texture for plane 0
texture for plane 0 is created (id=1)
creating texture for plane 1
texture for plane 1 is created (id=2)
creating texture for plane 2
texture for plane 2 is created (id=3)
timer: 1979711490, current thread: 0x8ee4d08, player thread: 0x8ee4d08
stopping demuxer thread...
stopping thread  QtAV::VideoThread(0xf2b6cf8) 
Queue is still empty
setInSize => calculating aspect ratio from converted input data(0.000000)
QtAV::VideoFormat(pixelFormat: -1 "" has alpha: false, channels: 0, planes: 32, bitsPerPixel: 0) 
pixel format changed: yuv420p =>  -1
Video thread stops running...
all avthread finished. try to exit demux thread<<<<<<
stopping demuxer thread...
all avthread finished. try to exit demux thread<<<<<<
User Interrupt: -> quit!
Demux thread stops running....
all audio/video threads  stopped...
demuxer thread emit finished.
avplayer emit stopped()

played for ~20 sec

wang-bin commented 9 years ago

the log seems fine. can you try a 1.8.0 installer?

ThatAIGeek commented 9 years ago

Hm, wait, I was trying it only in application and without recreating it in as short example as possible, so when I deleted all of widgets that where on top of player(a had 2 transparent or mostly transparent widgets), video suddenly started playing. Don't have a clue what caused that through. Maybe you can direct me towards possible source of problem?

ThatAIGeek commented 9 years ago

ok, I was able to locate the source of the problem, after call to player->lower() (where player is simplified version of simpleplayer from examples) I can see only black screen and no video.

ThatAIGeek commented 9 years ago

So apparently, GLWidgetRenderer2 doesn't work if it's not top level widget(. I'll investigate more tomorrow.

wang-bin commented 9 years ago

lower() makes the widget visually behind any overlapping sibling widgets

ThatAIGeek commented 9 years ago

Yes, I know that. The problem is that QGLWidget paints not in QT context, so if we paint on top of it in QT context, QT doesn't know how to fill background so it fills it with black color. I'll investigate it more and report if I'll fix this problem.

ThatAIGeek commented 9 years ago

To find more about this problem you can follow this links: http://stackoverflow.com/questions/11604062/semi-transparent-qwidget-over-qglwidget-strange-results http://www.qtcentre.org/threads/40335-QLabel-on-top-of-a-QGLWidget-background-issue

wang-bin commented 9 years ago

Can you give me an QGLWidget example that works for you?

ThatAIGeek commented 9 years ago

Didn't found it yet. I'll tell you when I'll find one

ThatAIGeek commented 9 years ago

I want have time to work on this issue until Wednesday, WidgetRenderer is fine for now. But so far, it looks like we need to call beginNativePainting and endNativePainting every time we are calling gl functions. There are notes about those in QtAV code as well(but only in comments).

Lexerize commented 9 years ago

Use QtAV::Direct2DRenderer in graphics scene gives me black screen, as well as QtAV::GraphicsItemRenderer with setOpenGL(true).

wang-bin commented 9 years ago

do not use direct2d. it can not work with qt components.

wang-bin commented 9 years ago

@Lexerys does videographicsitem example work?

Lexerize commented 9 years ago

Qt5.5.0, msvc 2015.

Videographicsitem example after enable openGl and loading h264 video:

"trying video decoder: FFmpeg..." "****Video decoder found:0x88d9760" "setCapture 0x239a90" "stream: 1, duration=-9223372036854775808 (-9223372036854775808 ms), time_base=0.001000" "codec: h264(H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10)" "Starting video thread..." "get av queue a/v thread = 0x0 0x87acb70" "AVClock started!!!!!!!!" "no frame could be decompressed: Error number 26097 occurred 0/26097" "invalid video frame from decoder. undecoded data size: 0" "no frame could be decompressed: Error number 368 occurred 0/368" "invalid video frame from decoder. undecoded data size: 0" "no frame could be decompressed: Error number 930 occurred 0/930" "invalid video frame from decoder. undecoded data size: 0" "no frame could be decompressed: Error number 2452 occurred 0/2452" "invalid video frame from decoder. undecoded data size: 0" "QtAV::VideoRenderer::setInSize => calculating aspect ratio from converted input data(1.777778)" "[ShaderManager] cache a new shader material type(8): gl material 8bit channel: 0, planar: 0, has alpha: 0, 2d texture: 1" "bind attribute: a_Position => 0" "bind attribute: a_TexCoords0 => 1" "glGetUniformLocation(\"u_Texture0\") = 23789568" "glGetUniformLocation(\"u_MVP_matrix\") = 23724032" "glGetUniformLocation(\"u_colorMatrix\") = 23920640" "glGetUniformLocation(\"u_opacity\") = 23986176" "glGetUniformLocation(\"u_c\") = 23855104" ASSERT failure in QVector::operator[]: "index out of range", file d:\programming\lib\qt\qt-5.5.0-msvc2015-x86\qtbase\include\qtcore../../src/corelib/tools/qvector.h, line 397 Debug Error!

stack trace: [image: Встроенное изображение 1]

2015-09-12 12:46 GMT+04:00 WangBin notifications@github.com:

@Lexerys https://github.com/Lexerys does videographicsitem example work?

— Reply to this email directly or view it on GitHub https://github.com/wang-bin/QtAV/issues/500#issuecomment-139740864.

Lexerize commented 9 years ago

QRectF VideoMaterial::mapToTexture(int plane, const QRectF &roi, int normalize) const { DPTR_D(const VideoMaterial);

// const qreal tex0W = d.texture_size[0].width(); const qreal tex0W = !d.texture_size.isEmpty() ? d.texture_size[0].width() : 0; // works fine ... }

But video aspect ratio always equal to window rect (try VideoRenderer::RendererAspectRatio, VideoAspectRatio).

2015-09-12 13:46 GMT+04:00 Alex Gavrilov al.vl.gavrilov@gmail.com:

Qt5.5.0, msvc 2015.

Videographicsitem example after enable openGl and loading h264 video:

"trying video decoder: FFmpeg..." "****Video decoder found:0x88d9760" "setCapture 0x239a90" "stream: 1, duration=-9223372036854775808 (-9223372036854775808 ms), time_base=0.001000" "codec: h264(H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10)" "Starting video thread..." "get av queue a/v thread = 0x0 0x87acb70" "AVClock started!!!!!!!!" "no frame could be decompressed: Error number 26097 occurred 0/26097" "invalid video frame from decoder. undecoded data size: 0" "no frame could be decompressed: Error number 368 occurred 0/368" "invalid video frame from decoder. undecoded data size: 0" "no frame could be decompressed: Error number 930 occurred 0/930" "invalid video frame from decoder. undecoded data size: 0" "no frame could be decompressed: Error number 2452 occurred 0/2452" "invalid video frame from decoder. undecoded data size: 0" "QtAV::VideoRenderer::setInSize => calculating aspect ratio from converted input data(1.777778)" "[ShaderManager] cache a new shader material type(8): gl material 8bit channel: 0, planar: 0, has alpha: 0, 2d texture: 1" "bind attribute: a_Position => 0" "bind attribute: a_TexCoords0 => 1" "glGetUniformLocation(\"u_Texture0\") = 23789568" "glGetUniformLocation(\"u_MVP_matrix\") = 23724032" "glGetUniformLocation(\"u_colorMatrix\") = 23920640" "glGetUniformLocation(\"u_opacity\") = 23986176" "glGetUniformLocation(\"u_c\") = 23855104" ASSERT failure in QVector::operator[]: "index out of range", file d:\programming\lib\qt\qt-5.5.0-msvc2015-x86\qtbase\include\qtcore../../src/corelib/tools/qvector.h, line 397 Debug Error!

stack trace: [image: Встроенное изображение 1]

2015-09-12 12:46 GMT+04:00 WangBin notifications@github.com:

@Lexerys https://github.com/Lexerys does videographicsitem example work?

— Reply to this email directly or view it on GitHub https://github.com/wang-bin/QtAV/issues/500#issuecomment-139740864.

wang-bin commented 9 years ago

@Lexerys try release build. or add return false; at GraphicsItemRenderer.cpp:70

Lexerize commented 9 years ago

With const qreal tex0W = d.texture_size[0].width(); release build crashes also With const qreal tex0W = !d.texture_size.isEmpty() ? d.texture_size[0].width() : 0; works fine, but aspect ratio...

Lexerize commented 9 years ago

Sorry, it's my bug, release works fine. But still need to be correct this code to work in debug mode.

LiamMartens commented 6 years ago

Encountered the same issue where the GLWidgetRenderer doesn't render when it is not the top level widget. Has this ever been fixed or worked around?