search

iamnpc / libyuv

Automatically exported from code.google.com/p/libyuv
0 stars 0 forks source link

ConvertFromI420 results in blue/purple image #26

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Call ConvertFromI420 with FOURCC_BGRA.
2.
3.

What is the expected output? What do you see instead?
The frame should be normal. The frame is blue/purple.

What version of the product are you using? On what operating system?
Possibly Windows bu confirmed on Mac OS X and iOS.

Please provide any additional information below.
Attached is a screenshot.

Original issue reported on code.google.com by J...@junglecat.org on 26 Mar 2012 at 2:53

Attachments:

GoogleCodeExporter commented 9 years ago 
It visually looks like the sort of issue you get if you convert to BGRA but 
view as ARGB.  The alpha is 255 and in the position of BLUE.
ARGB is usually the format you want on OSX and IOS.

Original comment by fbarch...@google.com on 28 Mar 2012 at 12:20

GoogleCodeExporter commented 9 years ago 
On Mac OS X I'm capturing with kCVPixelFormatType_32ARGB. On iOS I'm capturing 
with kCVPixelFormatType_32BGRA. Next I convert to I420. Finally I convert to 
FOURCC_BGRA and view using kCGBitmapByteOrder32Big.

I'm attaching or Mac OS X example of the problem. The lower left video stream 
doesn't undergo any conversion.

Original comment by J...@junglecat.org on 28 Mar 2012 at 5:04

Attachments:

GoogleCodeExporter commented 9 years ago 
It would be easier to deal with the 2 formats/platforms in 2 issues.
My understanding of the Apple ARGB types is they describe the channel order 
within a 32 bit integer.  That is how libyuv names work for ARGB, BGRA and ABGR.

kCVPixelFormatType_32ARGB means 'a' is the high byte of a register followed by 
r,g and b in the low byte.  In memory, being x86 and arm are little endian, B 
is the first byte, followed by G,R and then A.

In your second screenshot, it looks like the color channel order is correct but 
there is a conversion overflow/underflow?
The SSSE3 conversions from ARGB to I420 and back are supposed to saturate.
Can you disable sse2/ssse3, and if that works with C, isolate if its the 
ARGBToI420 or I420ToARGB?
You can use environment variables for quick tests.  export 
LIBYUV_DISABLE_SSE2=1 and export LIBYUV_DISABLE_SSSE3=1

Original comment by fbarch...@google.com on 28 Mar 2012 at 9:38

GoogleCodeExporter commented 9 years ago 
I think you should be using kCVPixelFormatType_32ARGB, FOURCC_ARGB and view 
using kCGBitmapByteOrder32Little?

Original comment by fbarch...@google.com on 29 Mar 2012 at 2:41

GoogleCodeExporter commented 9 years ago 
On iOS I am now using kCVPixelFormatType_32ARGB, FOURCC_ARGB and 
kCGBitmapByteOrder32Little but I still get the blue image for the converted 
frames. So now my problem is identical on iOS and Mac OS X. See attached.

Original comment by J...@junglecat.org on 29 Mar 2012 at 3:04

Attachments:

GoogleCodeExporter commented 9 years ago 
I'm currently re-architecting my code so if you'd like to close out this issue 
that's fine. If I discover this is still an issue I'll re-open.

Original comment by J...@junglecat.org on 1 Apr 2012 at 5:25

GoogleCodeExporter commented 9 years ago 
Re dots in comment 2.
We're seeing people report that effect when using Flash 11 on OSX.  But not on 
Windows or with Flash 10.

Re blue in comment 5
Just an endian issue with one of the functions.  libyuv is consistent 
internally.
If it turns out theres a bug, I'll certainly fix it, but I think its just 
confusion between OSX and libyuv naming.

Re comment6
For the camera end of things, you should normally capture YUV or MJPG.  Usually 
'yuvs' or '2yuv' on OSX and nv12 for iOS.
For rendering I420 or NV12 can be rendered with a shader, but converting to 
ARGB is easier.

Original comment by fbarch...@google.com on 5 Apr 2012 at 9:13

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
I've now implemented the renderer for Windows and I get blue also. So for Mac 
OS X, iOS and Windows any frame data that is passed through the converter is 
blue when rendered. I capture using MEDIASUBTYPE_RGB32 on Windows, 
kCVPixelFormatType_32BGRA on Mac OS X and kCVPixelFormatType_32BGRA on iOS.

Here is the flow for all platforms:
Capture(RGB32 or 
BGRA)->Convert(I420)->Encode(VP8)->Network(RTP)->Decode(VP8)->Convert(BGRA)->Ren
der(Direct2D or OpenGL)

If I bypass Encoding/Decoding the result is the same so I've confirmed it's not 
a codec issue.

I no longer get the "blue dots" but the effect is consistent across all three 
platforms.

Capturing in yuvs or 2yuv or nv12 would require re-architecting again and I'm 
not certain of the benefit.

What I do not know is is the blue because of the conversion "to" or "from". I'm 
about to re-route the flow to determine this.

Original comment by J...@junglecat.org on 5 Apr 2012 at 10:41

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
I deleted my last comment because I have further information. I've confirmed 
that the blue image is being caused by the "conversion to I420". Just after the 
conversion I wrote a frame to disk and viewed it with GLYUVPlay and it is 
indeed blue.

Original comment by J...@junglecat.org on 6 Apr 2012 at 12:41

GoogleCodeExporter commented 9 years ago 
Also I've been able to confirm that "conversion from I420 to BGRA" results in 
blue frames via the same method.

So any conversion to/from I420 to/from BGRA is blue.

Original comment by J...@junglecat.org on 6 Apr 2012 at 1:04

GoogleCodeExporter commented 9 years ago 
For libyuv the type you want is ARGB, not BGRA.
MEDIASUBTYPE_RGB32 on Windows is equivalent to libyuv FOURCC_ARGB
Use ARGBToI420 or ConvertToI420(..., libyuv::FOURCC_ARGB) to convert to I420.
Use I420ToARGB or ConvertFromI420(..., libyuv::FOURCC_ARGB) to convert to ARGB.

The ARGB format names in libyuv describe the order of channels within a 32 bit 
pixel.  A is the most significant 8 bits in a 32 bit pixel, followed by R, G 
and B in the least significant bits.  In memory they are stored the opposite 
order.

The advantage of capturing yuv formats (ie YUY2/yuvs) would be faster 
capture/conversion, when going to I420.

Original comment by fbarch...@google.com on 9 Apr 2012 at 4:29

GoogleCodeExporter commented 9 years ago 
Thanks for your help the problem is resolved on all platforms. I do find it odd 
that kCVPixelFormatType_32BGRA is actually ARGB but I understand your 
explanation.

I also understand that there would be less conversions required if YUV was used 
along most code paths.

Thanks Again!

Original comment by J...@junglecat.org on 10 Apr 2012 at 1:00

GoogleCodeExporter commented 9 years ago 
See Also include/libyuv/video_common.h for a list of sites that document the 
formats.

http://www.fourcc.org/yuv.php                                                   

http://v4l2spec.bytesex.org/spec/book1.htm                                      

http://developer.apple.com/quicktime/icefloe/dispatch020.html                   

http://msdn.microsoft.com/en-us/library/windows/desktop/dd206750(v=vs.85).aspx#n
v12

Original comment by fbarch...@google.com on 10 Apr 2012 at 6:13