Interlacement mismatch between FFV1 and MKV

[savcampbell]

When using vrecord to capture analog NTSC video to mkv/ffv1, I am having an issue where the interlacing in the file looks off. FFV1 metadata is showing the video is interlaced bottom field first, while mkv metadata is showing the value for FieldOrder as 14 ("Interlaced with top field displayed first and bottom field stored first"). The FFV1 and mkv metadata seem to contradict each other here. The mkv FieldOrder should probably be 9 ("Interlaced with bottom field displayed first and top field stored first") in this instance.

For more on this issue, see @dericed 's ticket here: http://trac.ffmpeg.org/ticket/6577#ticket

Perhaps vrecord can be edited to incorporate a script that addresses this issue?

[kieranjol]

This is really interesting, thanks for pointing it out. Looks like we have the same flipping issue for our PAL material. I replied to that ticket as it seems that this affects MOV as well as MKV?

[kieranjol]

Just for reference, the v210/mov file here is straight from Media Express with no further re-encoding, and is quite well described: https://archive.org/details/description_field_reel_field_scene_field_take_field_angle_field ffmpeg -i shows
Stream #0:0(eng): Video: v210 (v210 / 0x30313276), yuv422p10le(smpte170m/bt470bg/bt709, top coded first (swapped)), 720x576, 221184 kb/s, SAR 16:15 DAR 4:3, 25 fps, 25 tbr, 25k tbn, 25k tbc (default) It also has a value a value of 9 stored in the fiel atom: mediainfo --Details=1 description_field_reel_field_scene_field_take_field_angle_field.mov' | grep 13813 F13813 detail: 9 (0x09)

It would seem that 14 would be more appropriate here

14 – T is displayed earliest, B is stored first in the file

So I'm not sure what's going on here. Is Media Express also showing incorrect metadata?

[dericed]

That's interesting. Could you use Media Express with mov and prores (since both should document interlacement). I guess for a thorough audit we should ensure that these all match:

• Container interlacement: mkv:FieldOrder or mov:fiel
• Visual appearance: Output of ffmpeg -i INPUT -vf idet -f null -
• Encoding interlacement flags: Output of ffprobe test.mkv -show_entries frame=interlaced_frame,top_field_first (this is frame reporting, amirite?)
• MediaInfo summarization of interlacement for both container (General) and video.

[dericed]

If the output of vrecord and the output of Media Express match, then perhaps there's a misunderstanding somewhere.

[kieranjol]

Aye, I'll check those when I can. I can check what control room does too. That is what I was thinking, if ffmpeg, bmd, aja are all doing the same thing, is there a misunderstanding somewhere?

[bturkus]

Hi all,

I've been comparing NTSC v210/movs: Blackmagic Media Express and VRecord (captured with a Teranex 2D), and an AJA VTRXchange (captured with an IOXT). Here's what I've found:

Mediainfo General: scan order on all three is Bottom Field First Mediainfo --Details=1: field handling detail on all three is 14 (0x0E) Idet: potentially concerning diff? (apologies for the duration differences)

Vrecord

[image: Inline image 2]

Media Express

[image: Inline image 1]

VTRXchange

[image: Inline image 3]

QCTools confirms with idetm: Vrecord is blue (tff), Media Express pink (undefined), VTRXchange red (bff).

I can run other tests if needed (I didn't really understand the results of the ffprobe show_entries command)

On Thu, Aug 10, 2017 at 9:51 AM, kieranjol notifications@github.com wrote:

Aye, I'll check those when I can. I can check what control room does too. That is what I was thinking, if ffmpeg, bmd, aja are all doing the same thing, is there a misunderstanding somewhere?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/amiaopensource/vrecord/issues/170#issuecomment-321556899, or mute the thread https://github.com/notifications/unsubscribe-auth/AIkiXkwbzE3j0CZfOR9WkMqa4nkgK0ynks5sWwrEgaJpZM4OyquT .

--

Benjamin Turkus | The New York Public Library Assistant Manager for Audio and Moving Image Preservation

Barbara Goldsmith Preservation Division 40 Lincoln Center Plaza, New York, NY 10023 T. 212.870.1609 <(212)870-1609> | benjaminturkus@nypl.org http://www.nypl.org/preservation https://twitter.com/NYPLPreserve

Lifelong Learning | Advancing Knowledge | Strengthening Our Communities

[dericed]

@bturkus, no images. Can you run ffmpeg -i INPUT -vf idet -f null - to get idet summaries to share. IIRC, Savannah found that BFF frames dominated in her tests of vrecord captures.

[dericed]

@bturkus the ffprobe show-entires output shows booleans to say if the frame is interlaced or not and if the frame is top-field-first or not.

[bturkus]

my images! just noticed (my bad). media express vrecord vtrxchange

[kieranjol]

I just tried with prores/mov in AJA control room and that is showing 9 for the fiel atom value for a PAL transfer, which seems to be what you also see in vrecord and media express.

[Parsed_idet_0 @ 0x7fefe4800000] Repeated Fields: Neither:    14 Top:     0 Bottom:     0
[Parsed_idet_0 @ 0x7fefe4800000] Single frame detection: TFF:     0 BFF:     0 Progressive:     0 Undetermined:    14
[Parsed_idet_0 @ 0x7fefe4800000] Multi frame detection: TFF:     0 BFF:     0 Progressive:     0 Undetermined:    14

That test was just with colour bars so no real movement that could be picked up as interlaced. I can do more thorough tests on this when I have time.

[bturkus]

Could it be the prores? I was also using color bars, and my prores captures both came in as undetermined:

vrecord: [Parsed_idet_0 @ 0x7f9c33600000] Repeated Fields: Neither: 83 Top: 0 Bottom: 0 [Parsed_idet_0 @ 0x7f9c33600000] Single frame detection: TFF: 0 BFF: 0 Progressive: 0 Undetermined: 83 [Parsed_idet_0 @ 0x7f9c33600000] Multi frame detection: TFF: 0 BFF: 0 Progressive: 0 Undetermined: 83

media express: [Parsed_idet_0 @ 0x7ffd78507020] Repeated Fields: Neither: 38 Top: 0 Bottom: 0 [Parsed_idet_0 @ 0x7ffd78507020] Single frame detection: TFF: 0 BFF: 0 Progressive: 0 Undetermined: 38 [Parsed_idet_0 @ 0x7ffd78507020] Multi frame detection: TFF: 0 BFF: 0 Progressive: 0 Undetermined: 38

[kieranjol]

I don't think it's the prores, here's the same color bars that I captured directly to v210:

[Parsed_idet_0 @ 0x7fb848600000] Repeated Fields: Neither:    30 Top:     0 Bottom:     0
[Parsed_idet_0 @ 0x7fb848600000] Single frame detection: TFF:     0 BFF:     0 Progressive:     0 Undetermined:    30
[Parsed_idet_0 @ 0x7fb848600000] Multi frame detection: TFF:     0 BFF:     0 Progressive:     0 Undetermined:    30

and this is from some actual content later in the tape (it came from a film source so it shows up visually/with idet as progressive, though it's actually TFF:

[Parsed_idet_0 @ 0x7ff8ab800000] Repeated Fields: Neither:    42 Top:     0 Bottom:     0
[Parsed_idet_0 @ 0x7ff8ab800000] Single frame detection: TFF:     0 BFF:     0 Progressive:    40 Undetermined:     2
[Parsed_idet_0 @ 0x7ff8ab800000] Multi frame detection: TFF:     0 BFF:     0 Progressive:    41 Undetermined:     1

[bturkus]

Please disregard all of the information that I provided earlier. I just re-ran the test, creating 30 sec v210/movs of actual video content.

All were identical: bff in general Mediainfo, 14 in detailed, BFF:bottom coded_first, top displayed first in Mediaconch and QCTools, and BFF in idet-m.

Sorry to add to the confusion...

[dericed]

@bturkus in regards to your last comment, with what software (vrecord, ME)?

From the quicktime spec 14 is:

14 – T is displayed earliest, B is stored first in the file

From MKV spec:

14: Interlaced with top field displayed first and bottom field stored first

@jeromemartinez, any hints here?

[dericed]

A source of confusion here is that BFF is sometimes meaning Bottom Field coded First (such as in MediaInfoLib and sometimes means Bottom Field displayed First.

[bturkus]

Sorry for not being clearer. I compared three different 30 sec NTSC v210/mov captures: Media Express, Vrecord, and VTRXchange. All were 14.

And I just compared 2 PAL v210/mov captures, Media Express and Vrecord. Both 9

[dericed]

Also when I read https://developer.apple.com/library/content/technotes/tn2162/_index.html#//apple_ref/doc/uid/DTS40013070-CH1-TNTAG10-THE__FIEL__IMAGEDESCRIPTION_EXTENSION__FIELD_FRAME_INFORMATION for 14 it says:

if (detail == 14), then
    S(n)=n: buffer contains two fields woven together in spatial order
    case 2b: field containing line with lowest address is temporally later
    f (n mod 2 == 0), then T(n)=n/2+floor(height/2) else T(n)=floor(n/2)

( T(n) refers to the temporal (display) order of lines (n) and S(n) refers to the spatial order (storage)) from this reading I would presume that 14 means top field stored first and bottom field display first, but that seems to mismatch Apple's other citation of https://developer.apple.com/library/content/documentation/QuickTime/QTFF/QTFFChap3/qtff3.html which says:

14 – T is displayed earliest, B is stored first in the file.

[dericed]

@dwsinger any comments on https://github.com/amiaopensource/vrecord/issues/170#issuecomment-321646231 regarding the meaning of the fiel atom values?

[JeromeMartinez]

I already got such debate few years ago, with difficulties to understand specs compared to real world files. What I understood is that: 1 is for separate fields and T stored/displayed first 6 is for separate fields and B stored/displayed first 9 is for interleaved fields and T stored/displayed first 14 is for interleaved fields and B stored/displayed first

this corresponds to current FFmpeg behavior with FFV1 as FFV1 stores only interleaved fields.

with: ./ffmpeg -f lavfi -i mandelbrot -vf setfield=bff -vframes 1 -c:v ffv1 -level 3 -y test.mov You get with MediaInfo: Scan type : Interlaced Scan type, store method : Interleaved fields Scan order : Bottom Field First which seems the expected result (FFV1 stores interleaved fields and you want BFF)

with: ./ffmpeg -f lavfi -i mandelbrot -vf setfield=tff -vframes 1 -c:v ffv1 -level 3 -y test.mov You get with MediaInfo: Scan type : Interlaced Scan type, store method : Interleaved fields Scan order : Top Field First which seems the expected result (FFV1 stores interleaved fields and you want TFF)

Note: MI is missing "store method" line for Matroska, I'll add it (same way as with MOV).

From my point of view the spec is slightly misleading, and there no really stored vs displayed (I am curious of usefulness of storing first a field temporary later, but everything may exist...) but more separate fields / interleaved fields and TFF / BFF cases. I am not 100% sure, only empirical tests, and MediaInfo is implemented from theses empirical tests.

[dwsinger]

Ah, not quite. 1 and 6 are indeed 'planar' (all of one field before all of the other). They don't concern us. Both 9 and 14 are stored in spatial order (i.e. you could do terrible de-interlacing by simply displaying the buffer as a frame), and the 9 or 14 value tells you which field is to be displayed first.

9 – T is earlier than B. 14 – B is earlier than T

This is, afaik, what we do; the file format documentation appears to be, um, guilty of, well, wrongness. If you google "letters from the ice-floe" for Dispatch 19, this is all spelled out in the goriest of detail.

[kieranjol]

@dwsinger That's really helpful. So for the QuickTime and Matroska specifications, should the definitions of 9 and 14 be altered to specify that 9 – T is earlier than B. 14 – B is earlier than T , as both specs seem to say the opposite?

[dericed]

OK, so lots of mess, but IIUC correctly, then:

QuickTime File Format Specification states:

9 – B is displayed earliest, T is stored first in the file. 14 – T is displayed earliest, B is stored first in the file.

and this statement is wrong.

@dwsinger says in https://github.com/amiaopensource/vrecord/issues/170#issuecomment-321937668 that

9 – T is earlier than B. 14 – B is earlier than T

which in the terms of the QuickTime specification, I understand to mean:

9 – T is displayed earliest, B is stored first in the file. 14 – B is displayed earliest, T is stored first in the file.

Unfortunately the draft Matroska specification adopted the wrongness of the Apple QuickTime specification and states:

9 = Interlaced with bottom field displayed first and top field stored first. 14 = Interlaced with top field displayed first and bottom field stored first.

so the Matroska spec should be updated to correct the adopted wrongness.

Also the wrongness of 9 and 14 seems to not only apply to swapping top and bottom but also about storage order. From https://developer.apple.com/library/content/technotes/tn2162/_index.html#//apple_ref/doc/uid/DTS40013070-CH1-TNTAG10-THE__FIEL__IMAGEDESCRIPTION_EXTENSION__FIELD_FRAME_INFORMATION it seems that both 9 and 14 use the same storage order, so perhaps the Matroska draft should say:

9 = Interlaced with top field displayed first and fields stored as interleaved lines with the top field first. 14 = Interlaced with bottom field displayed first and fields stored as interleaved lines with the top field first.

If there's agreement, then I suggest the issues are moved from vrecord (which seems to set interlacement correctly) to move the issues to the Matroska specification (and nudges to @dwsinger to fix the Apple documentation).

Consensus?

[retokromer]

Indeed, let’s move it to Mastroska!

(We should also delve into ProRes in Matroska at some point. Possibly Vienna will be a good time to do so?)

[JeromeMartinez]

so perhaps the Matroska draft should say

You can't modify only half of the Matroska draft, so:

1 = Interlaced with top field displayed first and fields stored as planes (all of one field before all of the other) with the top field first. 6 = Interlaced with bottom field displayed first and fields stored as planes (all of one field before all of the other) with the bottom field first. 9 = Interlaced with top field displayed first and fields stored as interleaved lines with the top field first. 14 = Interlaced with bottom field displayed first and fields stored as interleaved lines with the top field first.

Note about FFmpeg: it looks like that current implementation of FFmpeg is using the QuickTime wording, but if we request a change to FFmpeg it will be conflicting with MXF because IIUC MXF has a method ("field dominance") for saying that the first stored field is not the first displayed field, and we remove such case in the definition, so I guess there will be a huge work on splitting things in FFmpeg (interleaved/planes, store order, display order: 3 things instead of current 2 things) before we can have right wording in it. Anyway, as the current FFmpeg implementation is compatible with the proposed definition, it is maybe not a short term issue.

I suggest the issues are moved from vrecord

Right, here is maybe not the right place for debating about that.

[kieranjol]

I'm relieved to have all this extra clarity, and yes, let's to Matroska!

[dericed]

I started a PR to fix field order definitions in Matroska at https://github.com/Matroska-Org/matroska-specification/pull/162 and commented on the issue in the cellar working group at https://mailarchive.ietf.org/arch/msg/cellar/1zIrVMgdqfQVXwaKLKDoFi8Qeyw.

[dericed]

Also related ffmpeg-devel patch at http://ffmpeg.org/pipermail/ffmpeg-devel/2017-August/214848.html.

[retokromer]

I guess the issue can be closed here on vrecord.