Partial solution for: head of stream frame droping - random frame droping - tail of stream frame dropping.

[GoogleCodeExporter]

What steps will reproduce the problem?

1. MPlayer playback of H.264

What is the expected output? What do you see instead?

the expected behavior is parsing ALL the frames.
running "mplayer -vc coreserve" on H.264 causes lots of drops.
Input of 1500 Frames, might result at 1400, 1350, 1450 frames.
(MPlayer will show all frames been parsed, but it won't match the amount 
of frames "-vo" receives.

What version of the product are you using? On what operating system?

MPlayer r27743 , coreavc-for-linux r72 , CoreAVC 1.8, CentOS 5
been verified on Multiple Linux Environments, 2 different users.

Please provide any additional information below.

I managed to play around with:
"DS_VideoDecoder.c" , and realized that: "sampleProcData.updated" == 0.

It seems that: "DS_Filter_CopySample" over "DS_Filter.c" - some of the 
times - being called TWICE, before ".updated" resets to "0" again.

which means: update=0;update=1;update=1;update=0; .. (update being 
assigned twice, the first time is AFTER the reset. the second time is 
BEFORE the reset, which means the following frame being dropped)

the dirty solution for the "initial" frame drops, and the "general" frame 
drops was: never resetting ".updated = 0", after the first time it being 
assigned as "1" .. keep it in that phase ... and it won't drop future 
frames nor Initial frames.

the Second issue I could not seem to see any resolution.. is the fact few 
frames being lost at the tail of the stream.. again - it varies.

Thanks a lot for the great work.
roo_.

Original issue reported on code.google.com by roori....@gmail.com on 12 Oct 2008 at 8:52

[GoogleCodeExporter]

Can you provide me with a sample that does this?
The reason for 'updated' is to deal with various types of packets.  In some 
streams
only partial NAL packets are sent and so CoreAVC wont send a return every time a
packet is received.  This can also happen for interlaced frames (though mplayer 
is
broken handling those)
The right way to solve this is likely to queue up frames in CopySampleand 
return the
1st frame in the queue (if any).  However, I'd need to see an example stream to
ensure this works properly.

Original comment by alannis...@gmail.com on 13 Oct 2008 at 1:08

• Changed state: Accepted

[GoogleCodeExporter]

http://iii.dontexist.net/pub/BigBuckBunny.mkv
H.264, encoded with x264.
MEncoder counted X frames... after using MPlayer it showed: "framecount=Y" 
about 10% 
of the frames were missing.

so far I couldn't find a sample who showed the exact same number of frames...

again, thanks a lot.
roo_.

Original comment by roori....@gmail.com on 13 Oct 2008 at 4:16

[GoogleCodeExporter]

BTW, just for the protocol - to clear things out.
you don't "have" to set the .updated value fixed.

instead of resetting ".updated = 0;" you may: -1;
and then, instead of setting it ".updated = 1;" , you may: +1;

this way, when this scenario happens:
updated=0; updated=1; updated=1; updated=0;

the result is obviously FALSE.

but this approach would be just fine:
updated -1; updated +1; updated +1; updated -1;

the result is TRUE, and its as-acceptable as "fixing" the state of updated to 1.

note, to make things easier you may just printf("\n updated=%d \n",updated);
while MPlayer playbacks, that way you can see perfectly where along the video 
stream, the frames been dropped.

roo_.

Original comment by roori....@gmail.com on 13 Oct 2008 at 2:55

[GoogleCodeExporter]

Do you actually have a problem you are trying to solve here?
I ran with the following patch, and it never triggered once despite the frame 
counts
in and out being different.
This is due to how mplayer demuxes the video and builds frames (basically it 
isn't
sending a complete frame all the time.
The 'fix' you have would just send the same frame multiple times in this case, 
which
isn't really what you want to do.

So in my mind the premise of this ticket is wrong (dshowserver will not 
necessarily
send one output for every input).  If there is a real display issue or you can 
show a
case where CopySample actually gets called multiple times for a single 'Receive'
call, then I'll be interested, as that is something I can work on fixing.
Please run with the applied patch and let me know if you find a stream which 
triggers
the new code.

Original comment by alannis...@gmail.com on 13 Oct 2008 at 6:34

Attachments:

• multiple_frames.patch

[GoogleCodeExporter]

"Do you actually have a problem" <-- MPlayer reads 1750 frames while using lavc 
- it 
actually exports 1750 frames.
while using -vc coreserve - it exports about 1600-1650 frames.

you should replace: "sampleProcData.updated = 0;"
with: "if (sampleProcData.updated) { sampleProcData.updated += -1; }"

here is a simple way to see the misbehavior:
1. download 50 samples of PNGs:
http://iii.dontexist.net/pub/bbb50samples.zip

2. encode them into 1.x264:
mencoder mf://*.png -mf fps=24:type=png -o 1.x264 -ovc x264 -x264encopts crf=30

3. run mplayer without coreserve, and export to PNGs again:
mplayer 1.x264 -vo png

result: now you will see 50 samples of PNGs, as you should.
50 x parsed, 50 x viewed.

4. now try coreserve:
mplayer 1.x264 -vo png -vc coreserve

result: you will never have your 50 frames back.
50 x parsed, ~40 x viewed.

with my "fix" .. you will get very very close, you will lose only few frames at 
the 
end of the stream. (mostly less than 10 frames)

but with the previous misbehavior - you would lose frames all across the 
stream... 
LOTS of frames.

hope this way to re-produce would show you exactly the previous misbehavior, 
and how 
my "fix" manage to greatly improve it.

with my "fix":
1. now it won't drop the initial frames.
2. it won't drop frames along the stream.

the current issue: few of the frames positioned at the END of the video stream, 
being chopped off.

Thanks a lot, roo_.

Original comment by roori....@gmail.com on 13 Oct 2008 at 11:54

[GoogleCodeExporter]

You haven't answered my question:
Do you actually see an issue when watching movies?
Your fix is wrong, and there is nothing to fix in your test case.  It is just 
how
CoreAVC behaves.
I didn't forget the 'updated -= 1' The patch wasn't supposed to fix the 
problem, but
instead show that there is no condition in which a frame gets dropped (i.e 
CopySample
called multiple times for a single input frame)

Here are the steps I ran to show what is going on:
1) create 1000 images with the numbers 0 - 999
convert xc:black -resize 800x600! blank.ppm
perl -e 'foreach $i (0..999) { $s = sprintf "%03d", $i; system "convert 
blank.ppm
-depth 8 -type TrueColor -gravity Center -pointsize 500 -stroke white -fill blue
-draw \"text 0,0 \\\"$i\\\"\" test$s.png";}'

2) encode using mencoder:
mencoder mf://*.png -mf fps=24:type=png -o 1.x264 -ovc x264 -x264encopts crf=30

3) decode using ffh264:
mkdir ffh264; cd ffh264
mplayer -vc ffh264 -vo png ../1.x264
cd ..

4) decode using dshowserver:
mkdir dshow; cd dshow
mplayer -vc coreserve -vo png ../1.x264
cd ..

5) Now check the frames:
The following should show that ffh264 does indeed save all 1000 frames
display ffh264/00000001.png
display ffh264/00001000.png
In all cases the number on the screen should be one less than the file number

Using dshowserve we only got 993 frames (note this value may vary depending on 
your
system), but:
display dshow/00000001.png
display dshow/00000993.png
shows that the same rule holds (number on screen is one less than file number)

This means that coreavc has 7 frames queued in RAM, but hasn't dropped any.  It 
just
has output lagging input by 7 frames.  You can never get the last 7 frames back
because mplayer won't ask for them.

Original comment by alannis...@gmail.com on 14 Oct 2008 at 3:13

[GoogleCodeExporter]

"Do you actually see an issue when watching movies?"
No, my eyes are not sharp enough to spot 10% loss of frames.

I do 'see' an issue while....
1. extracting frames.
2. exporting to PNG sequences.
3. while piping YUV4MPEGs.
4. while TRANSCODING.
etc etc... any task which requires a solid input.

BigBuckBunny.mkv contains: 14314 Frames.
I ran multiple tests on BigBuckBunny.mkv, WITHOUT my 'fix':
"mplayer -vo null BigBuckBunny.mkv -ac null -ao null -vc coreserve -benchmark"

1. framecount=1356
2. framecount=1281
3. framecount=1407
4. framecount=1342
5. framecount=1349
6. framecount=1294
7. framecount=1344

this are the results, WITH my 'fix':
1. framecount=14304
2. framecount=14304
3. framecount=14304
4. framecount=14305
5. framecount=14304

WITHOUT the 'fix' the sequence of the PNGs was not according to the sequence of 
the 
source.

PNG: 10000, never matched the source: 10000th frame.

WITH my 'fix', the sequence of the PNGs was IDENTICAL to the sequence of the 
source:
PNG: 14300, matches source: 14300th frame.

all the frames from 1-14300 matches 100% to the source: 1-14300.

anyhow,
I'm going to keep using my 'fix', as it works for my needs.
I thought you might be interesting with the solution,
unfortunately You can't seem to reproduce it.
I can - on several environments, and another fellow managed to reproduce it.

I was just wondering if there is a solution for the last few frames which never 
being extracted,
according to you - nope.

Thanks a lot for the good work, and your time.
roo_.

Original comment by roori....@gmail.com on 14 Oct 2008 at 8:14

[GoogleCodeExporter]

One more issue, another fellow been using my 'fix',
and commented its working - but seeking seems faulty.

I managed to fix it, here are all the "differences" from the latest git:
http://iii.dontexist.net/DS_VideoDecoder.c
http://iii.dontexist.net/DS_Filter.c

Applied Modifications:
file: DS_VideoDecoder.c
add new line#390: "sampleProcData.updated = 0;"
replace line#451: "sampleProcData.updated = 0;"
with: "if (sampleProcData.updated) { sampleProcData.updated += -1; }"

file: DS_Filter.c
replcae line#122: "pData->updated = 1;"
with: "pData->updated += 1;"

thats it, now it will process most all the frames, and won't cause the seeking 
issues.
(as far as I could see.)

roo_.

Original comment by roori....@gmail.com on 14 Oct 2008 at 9:38

[GoogleCodeExporter]

Well, the problem turns out to be CoreCodec 1.8 which I hadn't tried before.
Version 1.7.0 works properly, but 1.8.0 triggers the problem you are having.
Now that I can reproduce it, I will come up with a proper fix.  Again, you 
patch does
not do the right thing and I don't recommend using it.  You'd be better off 
using
CoreAVC 1.7.0 which seems to work properly, until I'm done.

Original comment by alannis...@gmail.com on 15 Oct 2008 at 1:07

[GoogleCodeExporter]

Actually, I'm restricted to 1.8, due to the requirement of the new 4:4:4 
profile.

Happy to hear that you can reproduce it and approach the difficulty better :)

I have not given up on the last few frames flaw, still trying.
(its Kinda useless to work with Lossless streams that you can't manipulate them 
and 
actually process ALL of thier frames.)

roo_.

Original comment by roori....@gmail.com on 15 Oct 2008 at 2:33

[GoogleCodeExporter]

I've applied a fix in R73 which should give the same performance in 1.7.0 and 
1.8.0
If you find a solution to losing the last few frames, feel free to open a 
seperate
ticket, but it is really just an incompatibility between mplayer and CoreAVC, 
and the
fix is likely to be in mplayer itself (send a bunch of null packets to try to 
recover
the last few frames for instance), though there may be something that can be 
done in
the dshowserver, I'm not really sure.

Original comment by alannis...@gmail.com on 15 Oct 2008 at 2:55

• Changed state: Fixed