Add support to avfvideosrc to output I420

[superdump]

If avfvideosrc can do an optimized conversion to I420 using a GL shader, software conversion to I420 when using VP8 could be entirely avoided.

[ikonst]

Were AVFoundation to offer I420 natively on iOS, would that be kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange or something else?

Reference: https://developer.apple.com/library/mac/documentation/QuartzCore/Reference/CVPixelFormatDescriptionRef/index.html#//apple_ref/doc/constant_group/Pixel_Format_Types

[ikonst]

@ijsf I think VP8 performance is your pet peeve.

On iPhone 5s, camera offers the following 3 formats:

FourCC	kCVPixelFormatType	GST_VIDEO_FORMAT	Description
420v	420YpCbCr8BiPlanarVideoRange	NV12	Bi-Planar Component Y'CbCr 8-bit 4:2:0, video-range (luma=[16,235] chroma=[16,240])
420f	420YpCbCr8BiPlanarFullRange	???	Bi-Planar Component Y'CbCr 8-bit 4:2:0, full-range (luma=[0,255] chroma=[1,255])
BGRA	32BGRA	BGRA	32 bit BGRA

So I understand I420 has 3 planes, whereas 420v and 420f have 2 planes, so unpacking is needed :(

[ikonst]

I naively tried ./gst-launch-1.0 avfvideosrc ! glcolorconvert ! gldownload ! vp8enc but got an endless stream of:

ERROR              glconvert gstglcolorconvert.c:1497:_do_convert:<glcolorconvert0> input must be GstGLMemory

[ikonst]

After fixing the code a bit — in particular, this: https://github.com/ikonst/gst-plugins-bad/commit/953e89763af12ee9ced80eabdd9145cd7be5a636 now gst-launch-1.0 avfvideosrc ! glcolorconvert ! gldownload ! vp8enc ! vp8dec ! osxvideosink works, but there are weird artifacts.

Might have something to do with @alessandrod 's fake-RGBA-that's-really-NV12 in Core Video textures?

[ijsf]

@ikonst I420, 420v and 420f are virtually the same (Y, Cb/U, Cr/V), as far as I know.

Actually, the artifacts look like some sort of stride/offset thing. Notice how the pixels are fairly far apart from each other in terms of linear addressing.

Also see: http://avisynth.nl/index.php/I420

[ikonst]

From what I understand, I420 is planar (i.e. kCVPixelFormatType_420YpCbCr8Planar) whereas 420v=NV12 and 420f is bi-planar (meaning Y is on its own plane and the U/V are interleaved).

[ikonst]

The artifacts are more complicated. Sometimes it's the dots you're seeing, but once in a while the whole image flashes green.

[superdump]

NV12 is also biplanar. The difference between 420v and 420f is the range of the YUV component samples. I think that is dependent on the colour standard used, don't remember what it's called. BT.601, Rec.709 stuff like that.

[ikonst]

That's exactly what I've said :)

NV12 (= 420YpCbCr8BiPlanarVideoRange = '420v') is bi-planar. I420 is planar (i.e. 3-planar).

Turning NV12 into I420 is a matter of de-interleaving YYYUVUVUV into YYYUUUVVV.

[superdump]

Ah, sorry. I read 420v=NV12 and 420f is biplanar as separate statements. :smile: