[Bug] avc ctts problems [sf#23]

[rbouqueau]

Reported by b-o-n-d on 2005-04-04 10:27 UTC i have now been able to calculate the cts/dts values of the .mp4 files created with mp4box i first calculated the dts/cts from .mp4 files produced by nero and moonlight and than remuxed the avc stream to .mp4 with mp4box and mp4creator and checked again the dts/cts

i found the following problems:

1) independant of the used number of max. cons. b- frames, mp4box sometimes uses the same cts value twice, which is obviously not correct, and also not there in the original source .mp4 and also not in the mp4creator .mp4 i could reproduce this on all three samples i tested. apart from that the nero samples didnt show another problem

2) i tested one moonlight sample i have and i, also additionally to 1), found another issue which was that mp4box seemed to place some cts values to the wrong dts (i assume mp4box is wrong as neither the original moonlight nor the remuxed mp4creator (which are identical btw) show these combinations)

for moonlight/mp4creator: dts 2064000, cts 2064000 dts 2080000, cts 2096000 dts 2096000, cts 2128000 dts 2112000, cts 2112000 <- ! dts 2128000, cts 2160000

for mp4box: dts 2064000, cts 2064000 dts 2080000, cts 2112000 <- ! dts 2096000, cts 2096000 dts 2112000, cts 2128000 dts 2128000, cts 2160000

this moved lines dont happen regularily

3) in the last line of the moonlight sample i also noticed that mp4box used a totally different value for the cts than moonlight/mp4creator (which also used the same), but not simply a value twice as described under 1):

moonlight/mp4creator: dts 35264000, cts 35280000

mp4box: dts 35264000, cts 35296000

---

i have uploaded some nero samples, showing 1), here: http://www.8ung.at/bond/avc\_ctts.7z

i havent tested arbitrary frame coding orders, but maybe this is also somehow related to this issue reported by algern0n: http://sourceforge.net/tracker/index.php? func=detail&aid=1162106&group_id=84101&atid=571741

[rbouqueau]

Commented by jeanlf on 2005-04-04 16:19 UTC Logged In: YES user_id=815447

Not sure which version of MP4Box you are using, I can't get exactly the same results here. Anyway there was indeed a pb, and indeed related to the other open bug on avc reordering. This should now be fixed on CVS.

Thanks for the samples, they helped debugging these issues. The extract-reimport of the nero file gives me the same DTS/CTS (scaled 1000) now. BTW, regarding MP4Creator, it uses negative CTS offset which is strictly forbidden in IsoMedia spec.

[rbouqueau]

Updated by jeanlf on 2005-04-04 16:19 UTC

• status: open --> closed

[rbouqueau]

Commented by b-o-n-d on 2005-04-04 19:21 UTC Logged In: YES user_id=786427

i used a cvs compile from 27 march, strange that you couldnt reproduce the nero problems?

i will check with recent cvs

[rbouqueau]

Commented by b-o-n-d on 2005-04-04 19:34 UTC Logged In: YES user_id=786427

i totally forgot, the moonlight sample i talked about is available here: http://www.moonlightcordless.com/clips/mp4/Belukha512x384 _1slice.mp4

[rbouqueau]

Commented by jeanlf on 2005-04-05 08:24 UTC Logged In: YES user_id=815447

I just tested couple of files from moonlight, I believe the AVC parsing on CVS should now be fixed as I couldn't get any of the previous issues with CTTS construction.

[rbouqueau]

Commented by b-o-n-d on 2005-04-05 20:03 UTC Logged In: YES user_id=786427

first of all: indeed all my reported bugs have been fixed with recent cvs :)

about the -dts option: a) which errors is this option able to detect exaclty? i noticed that my bugs 2) and 3) arent detected as being wrong by the dts option (bug 1) is detected) 2) and 3) were indeed bugs in mp4box, as i could reproduce them again with an older version of mp4box and now, with latest cvs, arent there anymore

b) i dunno if it makes any sense and i am not sure, but i think i and p-frames need to have the same value for dts and cts, right? maybe, but only if its easily doable, i would be nice to also display what kind of frame a sample is (i/p/b) of course with arbitrary frame coding orders in avc p-frames dont need to have the same dts/cts neither, but for asp it would maybe be a nice feature

thanks a lot for all your great work :)

[rbouqueau]

Commented by jeanlf on 2005-04-05 20:51 UTC Logged In: YES user_id=815447

'-dts' only checks CTTS table consistency, it's not a media aware process so you won't see I/P/B/whatever frame/slice type information there. What it does is check that for

• a given DTS is used once and only once in the media
• a given CTS is used once and only once in the media (old MP4Box bugs)
• a given sample CTS is always greater than or equal to this sample DTS (mp4creator muxed file don't always pass this test).

These tests are useless when no re-ordering occurs (no ctts table).

Regarding I and P frames, the answer is no: since you must have CTS>=DTS for all samples, this implies that for a stream with B-frames, only these B-frames (non-ref ones for AVC) will have DTS=CTS. All other frame/slices will need at least one frame delay. So the only case where DTS=CTS for I and P in video is when you don't have bidirectional prediction.

And more generally regarding "-dts" option, this is really just a quick debug tool, I doubt I'll add anything else to it ...

[rbouqueau]

Commented by b-o-n-d on 2005-04-05 21:50 UTC Logged In: YES user_id=786427

i found another bug with mp4box' ctts on this sample of moonlight: http://www.moonlightcordless.com/clips/mp4/Belukha512x384 _535kbps.mp4 mp4box ctts is totally borked according to -dts when remuxing this avc stream from raw

btw this stream uses frames of 2 slices and 2 b-frames, but mp4box' ctts was correct on a 2 sliced frame sample from vss (which used only 1 b-frames).

btw what other moonlight samples do you have and where do you have them from?

[rbouqueau]

Commented by jeanlf on 2005-04-06 11:23 UTC Logged In: YES user_id=815447

indeed, this was due to the 2 b-slices per b-frame - should now be fixed on CVS.

Regarding samples, don't know if we're talking about the same moonlight (I'd say so), I checked: http://www.moonlight.co.il/download/clips/

[rbouqueau]

Commented by b-o-n-d on 2005-04-06 16:45 UTC Logged In: YES user_id=786427

"indeed, this was due to the 2 b-slices per b-frame - should now be fixed on CVS."

great!

"Regarding samples, don't know if we're talking about the same moonlight (I'd say so), I checked: http://www.moonlight.co.il/download/clips/"

ah i thought you have found some more moonlight .mp4 samples (i know only the two i linked to in this report)

[rbouqueau]

Commented by b-o-n-d on 2005-04-15 21:32 UTC Logged In: YES user_id=786427

"indeed, this was due to the 2 b-slices per b-frame - should now be fixed on CVS."

fixed indeed

[rbouqueau]

Updated by b-o-n-d on 2005-04-19 15:13 UTC

• status: closed --> open

[rbouqueau]

Commented by b-o-n-d on 2005-04-19 15:13 UTC Logged In: YES user_id=786427

tested new cts code:

A) AVC 1) 1 b-frames: correct 2) 2 b-frames: correct 3) 2 b-slices per b-frame (1b-frames) (vss): correct 4) 2 b-slices per b-frame (2b-frames) (http://www.moonlightcordless.com/clips/mp4/Belukha512x38 4_535kbps.mp4): wrong ctts of mp4box (mp4creator and moonlight have the same) 5) b-references/pyramid (5 b-frames): correct (compared x264's direct mp4 output with mp4box's muxed mp4) 6) p-references (mainconcept: http://www.8ung.at/bond/mc\_delay.zip): wrong (see http://sourceforge.net/tracker/index.php? func=detail&aid=1162106&group_id=84101&atid=571741)

B) ASP 1) 1 b-frames: correct 2) 2 b-frames: correct 3) 3 b-frames: correct

correct here means that mp4box gave the same result as either nero, mp4creator or moonlight

to sum it up: there are issues with multiple sliced frames and arbitrary frame orders without b-frames

[rbouqueau]

Commented by b-o-n-d on 2005-05-05 19:06 UTC Logged In: YES user_id=786427

i tested the sorenson avc encoder:

1) it seems that mp4box doesnt recognize b-references when multiple slices are used in the same stream (at least the info output of mp4box during importing doesnt mention it), as can be seen on the following sorenson avc streams: http://www.heise.de/ct/05/10/146/tabelle\_engel.shtml (only 900kbps and 750kbps samples of sorenson show this, as only them use multiple slices)

2) grab the 30 and 250kbps samples (also use b-references) and have a look at the ctts created by mp4box: lots of negative offsets written altough the two samples dont use multiple slices [btw i also cant reproduce the sorenson cts with mp4creator,
which should handle b-ref fine too, so maybe the streams itself is borked?]

[rbouqueau]

Commented by b-o-n-d on 2005-05-15 13:33 UTC Logged In: YES user_id=786427

ok lets structure the open issues:

1) normal multiple sliced frames:

fixed (tested on moonlight, videosoft and apple)

2) multiple sliced with b-references (sorenson):

problem (mp4box doesnt give the same cts as mp4creator) is still here with the sorenson 900 and 750kbps clips. mp4box also doesnt display the info that b-ref has been used)

3) negative offsets with sorenson b-refs (not multiple sliced)

fixed, but there seem to be problems at the end of the stream, as duplicate cts get used and one negative offset (same behaviour on both samples: 30 and 250kbps)

4) reordered p-frames

fixed (mp4box gives the same ctts as mp4creator, one that seems to make sense), but there also is a problem at the end of one stream i tested (and that was the only stream of mainconcept which showed this, the others i tested worked fine): mp4box uses a different cts for the last frame as mp4creator (dunno which tool is right tough) i uploaded the stream plus cts here: http://www.8ung.at/bond/mc\_delay\_lastframe-ctts.zip

[rbouqueau]

Commented by jeanlf on 2005-05-15 14:07 UTC Logged In: YES user_id=815447

I'll look at 2)

regarding 3), I noticed sorenson bitstream use a duplicated POC at the end of the steram, which indeed corrupts MP4Box CTS reconstruction. Not sure exactly how to handle this...

I'll check what happens with 4).

[rbouqueau]

Commented by jeanlf on 2005-05-16 12:19 UTC Logged In: YES user_id=815447

regarding 2), I'm not sure whether these streams use b-refs

I added a dirty hack to handle sorenson end of streams and make sure last frames CTS is not broken

[rbouqueau]

Commented by b-o-n-d on 2005-05-16 13:43 UTC Logged In: YES user_id=786427

ad 2) i think that the use b-refs because: a) the lower bitrate samples also use b-refs, so it would be more likely that the higher bitrates ones do that too instead of the opposite i guess!? b) mp4creator uses a cts of 2 for the first sample, which indicates b-ref afaik

ad 3) ok, dirty hacks are always fun ;)

ad 4) any findings? is this maybe fixed with the sorenson hack too?

[rbouqueau]

Commented by b-o-n-d on 2005-05-16 15:29 UTC Logged In: YES user_id=786427

ad 3) ok i dont get the duplicate cts anymore aso, but still the cts is different than the one produced by mp4creator dunno which one is right, tough

ad 4) still the same

[rbouqueau]

Commented by jeanlf on 2005-05-16 17:53 UTC Logged In: YES user_id=815447

2) Hmm really weird, I can't get any out-of-order B slices when parsing these 2 streams. I'd say these two don't use b- refs, OR there's another bug/spec subtility somewhere... regarding MP4Creator CTSs, the offset of 2 could be picked due to this pb at end of stream...

3) don't know either, I first need to find out if such streams are correct or not...

4) Can't really help, MP4Creator sets negative CTS offsets everywhere on this stream...

[rbouqueau]

Updated by jeanlf on 2005-05-16 17:53 UTC

• status: open --> pending

[rbouqueau]

Commented by b-o-n-d on 2005-05-16 21:03 UTC Logged In: YES user_id=786427

"4) Can't really help, MP4Creator sets negative CTS offsets everywhere on this stream..."

hm, here it gives exactly the same cts as mp4box except for the last stream. that is version 1.2.18, which version did you use?

[rbouqueau]

Updated by b-o-n-d on 2005-05-16 21:03 UTC

• status: pending --> open

[rbouqueau]

Updated by b-o-n-d on 2005-05-17 10:51 UTC

• status: open --> pending

[rbouqueau]

Commented by b-o-n-d on 2005-05-28 14:15 UTC Logged In: YES user_id=786427

"2) Hmm really weird, I can't get any out-of-order B slices when parsing these 2 streams. I'd say these two don't use b- refs, OR there's another bug/spec subtility somewhere... "

ok, with a latest not yet released patch x264 is able to combine multislices and b-refs and mp4box works great and these samples, outputting the same cts as mp4creator it also reports that b-refs have been used correctly

so i assume that the sorenson streams are maybe somehow borked (it seems they use a variable number of slices per frame, especially on i-frames)

[rbouqueau]

Updated by b-o-n-d on 2005-05-28 14:15 UTC

• status: pending --> open

[rbouqueau]

Updated by jeanlf on 2005-06-03 12:22 UTC

• status: open --> closed-fixed