trying to support avdevice

[sherpya]

I'm trying to support avdevice via forcing input format, I've made a branch https://github.com/sherpya/QtAV/tree/avdevice but the video lags, video thread spams a lot of "delay x/x" and after a while the application crashes

in readFrame():

memcpy(encoded.data(), packet.data, encoded.capacity());

crashes even if it just reads packet.size from packet.data

maybe the frame decoding fails?

I suspect QtAV is not aware of the fps (for the delay) and it misses a check somewhere (for the crash)

any idea?

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/5021321-trying-to-support-avdevice?utm_campaign=plugin&utm_content=tracker%2F307703&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F307703&utm_medium=issues&utm_source=github).

[sherpya]

I've removed the branch, now on master https://github.com/sherpya/QtAV

[wang-bin]

i have no idea now. i'm travelling and i will try you code if i have time

[sherpya]

after some tests looks like if i divide pts by AV_TIME_BASE (1000000) in AVDemuxer::readFrame() the video works fine, but obviously breaks file based media

btw delay in the video thread is -0.523333 and videothread spams the log

here the original log:

AVPacket.pts=6163.871613, duration=0.033333, dts=6163871613 AVPacket.pts=6163.904381, duration=0.033333, dts=6163904381 AVPacket.pts=6163.937167, duration=0.033333, dts=6163937167 AVPacket.pts=6163.969920, duration=0.033333, dts=6163969920 AVPacket.pts=6164.002687, duration=0.033333, dts=6164002687 AVPacket.pts=6164.035455, duration=0.033333, dts=6164035455 AVPacket.pts=6164.068241, duration=0.033333, dts=6164068241

the input is video4linux2 camera 30 fps (0.03)

I'm also experimenting some crashes in readFrame() (using the changes above) in decoder's memcpy, other various crashes are in convertTo() of videothread, looks like there is not enough data in the packet (maybe decoding fails and packet.size has the previous frame's value)

e.g.:

    QByteArray encoded;;
    encoded.reserve(packet.size + FF_INPUT_BUFFER_PADDING_SIZE);
    encoded.resize(packet.size);
    // also copy  padding data(usually 0)
    memcpy(encoded.data(), packet.data, encoded.capacity());

packet.size is correct but memcpy crashes

sorry for merging bugs but they are reproduced together

[sherpya]

I've found something, in AVClock::updateValue the clock pts is only updated if the clock type is audio, my source has no audio, removing the conditional make the sync work, but I still get spam because the delay is 23918.6 23918.7 or 23918.8, it's a pts diff, the value should be already scaled with time_base

about the crash I suppose it happens when dropping frames since it happens if I busy the ui (moving resizing)

[sherpya]

sorry ignore my comment about AVClock I just got confused ;(

[wang-bin]

it seems that the timestamp of video frame from camera is the time since compute startup. the stream start time is the time since compute startup too. So I have to set an initial value to AVClock. I will push the code later

Another issue is opengl displays nothing. you can workaround with "player menu->colorspace->engine->libswscale" temporary

[wang-bin]

try the latest code

[wang-bin]

btw, i'd like to merge your avdevice code

[sherpya]

I have another commit and my master has other unrelated commits, I'll prepare a pull request

[sherpya]

hi, the latest code solves the pts problem, unfortunately it crashes on pause or when it drops frames (I think), AVDemuxer::readFrame() memcpy() reads uninit data even av_read_frame() returns 0 and packet values looks ok, I've tested zeroing avpacket and I see no difference when it crashes:

Flags=0x00000001 size=614400 pts=2760221898 dts=2760221898 0x7fd9ffaba000
Flags=0x00000001 size=614400 pts=2760280140 dts=2760280140 0x7fd9ffa24000
Flags=0x00000001 size=614400 pts=2760338360 dts=2760338360 0x7fda04c4f000

the last value is the ptr of data often it crashes in converTo() trying to access to same data pointer, it's reproducible e.g. using valgrind because slows down a lot

I've made a branch with only avdevice patch, I'll make a pull request in the meanwhile

[wang-bin]

I tested 3 days ago and it works fine. but it crashes today.

where is converTo()

[wang-bin]

try the latest code

I was using your master branch at commit f51a583f999405cb625cd999b8f6804a222fb141 and add AVClock patch(5b194f21bfca50332c348646a11d8df56354b985) and opengl patch (7f17a41ac413e1078f927128a4d388f51f11808a) then it works fine. But the latest code crashes. I have no idea now.

[wang-bin]

checkout 50c222eba1fe0e43ce1962624c4a8984b686f876 and create a new test branch, cherry-pick 5b194f21bfca50332c348646a11d8df56354b985 7f17a41ac413e1078f927128a4d388f51f11808a d59c1c1aa2cc4b2e3ef2a8052c99ea4f9ca56643 ea94283174d05b2570a585f57df0dbd077fecb5d and it works. I will find the reason later

[sherpya]

yes it does not crashes but after some start/stop it accumulates delay

[wang-bin]

the delay is fixed in a later commit 6c7790f

I will check the crash later

[sherpya]

better, but still a mess after some start stop, I can get more sync manually calling a seek (I use 600000 for example)

then the log spams video becomes slower. force reduce video delay delay -594.957014/3765.998817 delay -594.955195/3766.029817 delay -594.953888/3766.061817

I've also tested dshow on win32 and it lags a bit from start I suspect it needs the framerate to be set, params can be set by avoptions I have a patch for this but it needs to be cleaned, also I think it can be set somehow using the already present setOptions()

[sherpya]

I think there are some races when using external clock, mostly around AVClock::value(), the mutable _pts is assigned in value(), I think it's not a good idea. Perhaps I still have crashes while accessing to decoded packets even if packet size and packet data seem reasonable, maybe unrelated but somehow mitigated if using some additional locks

[sherpya]

btw sample player does not crash while the sample threadplayer crashes

[wang-bin]

simpleplayer can play avdevice without crash? i have no linux pc this week. i will continue next week

[sherpya]

simpleplayer does not crashes while playerthread crashes, I'm also testing a custom sequential qiodevice (with some changes on qtav) and it crashes after some frame on linux (running under valgrind the lives a bit more), on win32 it does no crashes, there is something different in qthread code between linux and windows It takes frame from a grabber device using proprietary sdk, currently on windows is very slow I still need to understand if my code in qiodevice sucks or there is a problem with framerate

[wang-bin]

change

        memcpy(encoded.data(), packet.data, encoded.capacity()); 

to

        memcpy(encoded.data(), packet.data, packet.size); 

then no crash. I will find the reason later. It's not a solution. some streams will not be decoded correctly. see #191

[wang-bin]

crash is fixed in 8bde621df909886a1d54073fec903f48c9a74b8d, please confirm it if ffmpeg>=2.1 or libav>=10, no data copy, just add a reference. If ffmpeg/libav is old, copy AVPacket.size(no crash for me) and set padding data to 0

[sherpya]

It works fine using your player:

player avdevice:video4linux2:/dev/video0

unfortunately after pressing stop and play again it crashes on Linux. I'm using ffmpeg (2.5.3).

Looks like there are no such problems on OSX.

here the bt

0x00007ffff468270a in ?? () from /usr/lib/x86_64-linux-gnu/libavdevice-ffmpeg.so.56
(gdb) bt
#0  0x00007ffff468270a in ?? () from /usr/lib/x86_64-linux-gnu/libavdevice-ffmpeg.so.56
#1  0x00007ffff21cdb0e in av_buffer_unref () from /usr/lib/x86_64-linux-gnu/libavutil-ffmpeg.so.54
#2  0x00007ffff2b86205 in av_packet_unref ()
   from /usr/lib/x86_64-linux-gnu/libavcodec-ffmpeg.so.56
#3  0x00007ffff7b5f596 in QtAV::Packet::operator=(QtAV::Packet const&) ()
   from /usr/lib/x86_64-linux-gnu/libQtAV.so.1
#4  0x00007ffff7b4ca4a in QtAV::AVDemuxer::readFrame() ()
   from /usr/lib/x86_64-linux-gnu/libQtAV.so.1
#5  0x00007ffff7b53ea0 in QtAV::AVDemuxThread::run() ()
   from /usr/lib/x86_64-linux-gnu/libQtAV.so.1
#6  0x00007ffff5def0de in ?? () from /usr/lib/x86_64-linux-gnu/libQt5Core.so.5
#7  0x00007ffff58a30a4 in start_thread (arg=0x7fffca8e2700) at pthread_create.c:309
#8  0x00007ffff4db5ccd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Also: pause screws playing and you should inhibit playback rate other than 1.0