Compile/Link fixes for libva < 2.0.0

[mighty-p]

Hi,

the following patch makes vaapidevice compile with earlier version of libva (for me it compiles with this with libva 1.7.0):

--- video.c.orig    2018-02-21 10:59:01.807698117 +0100
+++ video.c 2018-02-21 14:51:50.527644830 +0100
@@ -81,9 +81,9 @@
 #endif

 #include <va/va_x11.h>
-#if !VA_CHECK_VERSION(1,0,0)
-#error "libva is too old - please, upgrade!"
-#endif
+//#if !VA_CHECK_VERSION(1,0,0)
+//#error "libva is too old - please, upgrade!"
+//#endif
 #include <va/va_vpp.h>
 #ifdef USE_GLX
 #include <va/va_glx.h>
@@ -2291,9 +2291,11 @@
     setenv("DISPLAY", display_name, 1);

 #ifndef DEBUG
+#if VA_CHECK_VERSION(0,40,0)
     vaSetErrorCallback(VaDisplay, NULL, NULL);
     vaSetInfoCallback(VaDisplay, NULL, NULL);
 #endif
+#endif
     if (vaInitialize(VaDisplay, &major, &minor) != VA_STATUS_SUCCESS) {
    Error("video/vaapi: Can't inititialize VA-API on '%s'", display_name);
    vaTerminate(VaDisplay);
@@ -2426,7 +2428,7 @@
        // intel: NV12 is native format for H.264 decoded surfaces
    case AV_PIX_FMT_YUV420P:
    case AV_PIX_FMT_YUVJ420P:
-       fourcc = VA_FOURCC_I420;    // aka. VA_FOURCC_IYUV
+       fourcc = VA_FOURCC_IYUV;    // orig. VA_FOURCC_I420
        break;
    case AV_PIX_FMT_NV12:
        fourcc = VA_FOURCC_NV12;
@@ -4417,7 +4419,7 @@
            = frame->data[2][i * frame->linesize[2] + x];
        }
        }
-   } else if (decoder->Image->format.fourcc == VA_FOURCC_I420) {
+   } else if (decoder->Image->format.fourcc == VA_FOURCC_IYUV) {
        picture->data[0] = va_image_data + decoder->Image->offsets[0];
        picture->linesize[0] = decoder->Image->pitches[0];
        picture->data[1] = va_image_data + decoder->Image->offsets[1];

[mighty-p]

In addition, the following patch removes a flickering problem caused by a bug in libva<1.7.2 (it should not be needed anymore with 1.7.2 and higher):

--- video.c.orig    2018-02-21 10:59:01.807698117 +0100
+++ video.c 2018-02-21 14:51:50.527644830 +0100
@@ -3279,7 +3281,7 @@
            decoder->gpe_filters[decoder->gpe_filter_n++] = filter_buf_id;
        }
        break;
-       case VAProcFilterColorBalance:
+       case VAProcFilterColorBalance: break;
        Info("video/vaapi: enabling color balance filters");
        colorbalance_cap_n = VAProcColorBalanceCount;
        vaQueryVideoProcFilterCaps(VaDisplay, decoder->vpp_ctx, VAProcFilterColorBalance, colorbalance_caps,

[pesintta]

Support for libva < 2.0 was actually removed intentionally to encourage people to upgrade. Also many other unnecessary work-arounds for bugs in libva were removed from cluttering the code at the same time.

Therefore I'm reluctant to add back support for old libva versions. Do you have a reason why you cannot use libva-2.0 or greater?

[mighty-p]

The only reason is that Linux Mint 18.3 (which is the latest version) still comes with a pretty old libva. Other distros, e.g. Ubuntu, should have the same issue. For me it's easier to patch vaapidevice rather than updating a system-wide library (where I don't know if there are side effects on other software in the distro or on my ability to do a dist upgrade later). I can try if I can install a newer libva separately in /opt. Overall, I understand that it makes only very limited sense to work around old bugs in an outdated library :)

[dnehring7]

I do not see any problems by upgrading to libva 2.0. Perhaps, you can consider to set the minimum version to 2.1.0, which has some interesting features:

"*Bump VA-API version to 1.1.0 and libva to 2.1.0 Add API for multi-frame processing Add entrypoint VAEntrypointStats for Statistics Add data structures for HEVC FEI support Add new attributes for decoding/encoding/video processing Add new VPP filter for Total Color Correction Add blending interface in VPP Add rotation interface in VPP Add mirroring interface in VPP Add Chroma siting flags in VPP Add new color standard definitions Add new interface for exporting surface Add message callbacks for drivers to use"

Seems to be the new library version for Fedora 28.