PixelFormat invalid

[elieserdejesus]

Hi Daniel, your work with Juce and FFmpeg are amazing.

I'm testing your video engine and the VideoPlayer example play the audio (from a mp4 video) but the video frames are black (no video).

I changed the scaler output pixel format from AV_PIX_FMT_BGR0 to AV_PIX_FMT_RGB32 and the video is playing perfectly:

https://github.com/ffAudio/foleys_video_engine/blob/master/ReadWrite/FFmpeg/foleys_FFmpegReader.cpp#L87

I changed

scaler.setupScaler (videoContext->width,
                                videoContext->height,
                                videoContext->pix_fmt,
                                videoContext->width,
                                videoContext->height,
                                AV_PIX_FMT_BGR0);

to

scaler.setupScaler (videoContext->width,
                                videoContext->height,
                                videoContext->pix_fmt,
                                videoContext->width,
                                videoContext->height,
                                AV_PIX_FMT_RGB32);

I see AV_PIX_FMT_BGR0 is CUDA accelerated, right?

I'm testing in Windows 10, compiling in MSVC 2017 and running in a laptop with a builtin nvidia graphic card. I suppose CUDA is not working in my side.

I'm compiling with FFmpeg 4.2.2.

You think is possible adjust the scaler output pixel format to match user machine?

Thanks for your time, I really enjoy your work as a developer.

[ffAudio]

Thanks for the kind feedback and for making me aware of that issue. I didn't test too deeply on windows, so this slipped through the net. The code should figure out the native format of the Graphics backend, which was BGR0 on my mac. Do you happen to know, if AV_PIX_FMT_RGB32 will generally work on windows, so I could just bolt that in via #ifdef JUCE_WIN? Eventually a code is needed to probe the juce::Image class for the underlying format.

[elieserdejesus]

Do you happen to know, if AV_PIX_FMT_RGB32 will generally work on windows

I don't know if RGB32 is the "default" for windows, I see many other codes using AV_PIX_FMT_RGB24 instead AV_PIX_FMT_RGB32, so I'm still searching.

Eventually a code is needed to probe the juce::Image class for the underlying format.

I tried that, juce::Image has a getPixelFormat(), but the usefull values are just RGB and ARGB. Is not possible discover if the juce image is BGR0.

[elieserdejesus]

Hi again Daniel, I'm experiment with lib264 encoder, using just swscale (not avcodec) and this is working in Windows 10. I discovered my native format is AV_PIX_FMT_BGR24. I hope you have some insight.

    void convertImageToPicture(const juce::Image& image, x264_picture_t &picture)
    {
        auto juceNativePixelFormat = AV_PIX_FMT_BGR24; // windows native is little endian
        #if AV_HAVE_BIGENDIAN
            juceNativeImageFormat = AV_PIX_FMT_RGB24; // mac is big endian ?
        #endif

        scalerContext = sws_getCachedContext(scalerContext, 
            image.getWidth(), image.getHeight(), juceNativePixelFormat,
            width, height, AV_PIX_FMT_YUV420P, 
            SWS_FAST_BILINEAR, nullptr, nullptr, nullptr);

        juce::Image::BitmapData bitmap(image, 0, 0, image.getWidth(), image.getHeight());

        uint8_t* rgbPlanes[1] = { bitmap.data };
        int rgbLineSize[1] = { 3 * bitmap.width };

        sws_scale(scalerContext, rgbPlanes, rgbLineSize, 0, height, picture.img.plane, picture.img.i_stride);
    }