OpenCL surface sharing on Linux

[EmonLu]

Notice that sharing surface with OpenCL only support windows now. And all common operations about opencl are defined in structure OpenCLCtxD3D11.

While in MediaSDK sample, a base class OpenCLFilterBase and two OS-specific sub-classes OpenCLFilterDX11 / OpenCLFilterVA are defined which seems more clear.

When will OpenCL surface sharing on Linux enabled in VPL and will the OpenCLCtxD3D11 structure modified to separate common functions and OS-specific functions?

[jonrecker]

Thank you for your input. OpenCL surface sharing for Linux is currently under consideration, but there is not yet a timeline for this feature. Knowing that this would be useful for your application does help with prioritization.

The OpenCLCtxD3D11 structure is part of the example code and not VPL API, so applications are free to modify it. Probably the OCL surface sharing example code will be refactored when Linux support is added.

[EmonLu]

@jonrecker Thanks for your reply and I have modified the hello-sharing-x sample to implement OpenCL surface sharing on Linux referring to the code in MediaSDK.

I was trying to export a shared VAAPI surface from FrameInterface then use clCreateFromVA_APIMediaSurfaceINTEL function to create a OpenCL surface from the exported VAAPI surface. Main codes are as below:

mfxSurfaceHeader export_header = {};
export_header.SurfaceType      = MFX_SURFACE_TYPE_VAAPI;
export_header.SurfaceFlags     = MFX_SURFACE_FLAG_EXPORT_SHARED;
mfxSurfaceHeader *exportedSurfaceGeneral = nullptr;
mfxStatus stsSurf = encSurfaceIn->FrameInterface->Export(encSurfaceIn,
                                                         export_header,
                                                         &exportedSurfaceGeneral);
VERIFY(MFX_ERR_NONE == stsSurf, "ERROR: MFXMemory_ExportFrameSurface");
auto exportedSurface = reinterpret_cast<mfxSurfaceVAAPI *>(exportedSurfaceGeneral);
// create OpenCL surface from VAAPI surface 
cl_mem mem_a;
if (0 != oclCtx->CreateCLMemFromVASurface(oclCtx->GetOpenCLContext(), exportedSurface->vaSurfaceID, mem_a))
    return -1; 

cl_int OpenCLCtxVA::CreateCLMemFromVASurface(cl_context context, VASurfaceID surface, cl_mem &clMemA) {
    cl_int err;
    VASurfaceID* surf = &surface;
    clMemA = m_pfn_clCreateFromVA_APIMediaSurfaceINTEL(
        context,
        CL_MEM_READ_WRITE,
        surf,
        0,
        &err
    );
    if (err != CL_SUCCESS) {
        std::cerr << "Failed to create OpenCL memory object from VA surface: " << err << std::endl;
        return -1;
    }
    return err;
}

While clCreateFromVA_APIMediaSurfaceINTEL returns error code 18 which means invalid flags or plane. Then I checked the vaSurfaceID with codes belowing:

// check vaSurfaceID
VASurfaceStatus surfaceStatus;
VAStatus status = vaQuerySurfaceStatus(devCtx.GetVADisplay(), exportedSurface->vaSurfaceID, &surfaceStatus);

if (surfaceStatus == VA_STATUS_SUCCESS) {
    std::cout << "VASurfaceID is valid and its status is: " << surfaceStatus << std::endl;
} else {
    std::cerr << "VASurfaceID is invalid or cannot be queried. Error: " << surfaceStatus << std::endl;
}

which turns out to be VASurfaceID is invalid or cannot be queried. Error: 4. I also checked the vaapi surface in hello-sharing-vaapi-export and got the same return. Why the VASurfaceID is invalid? How can I create a OpenCL surface with the exported mfxSurfaceVAAPI? Or is there any better method?

typedef struct {
    mfxSurfaceInterface SurfaceInterface;

    mfxHDL vaDisplay;                           /*!< Object of type VADisplay. */
    mfxU32 vaSurfaceID;                         /*!< Object of type VASurfaceID. */
    mfxU32 reserved1;

    mfxHDL reserved[6];
} mfxSurfaceVAAPI;

[jonrecker]

Adding @mgonchar - would you expect vaQuerySurfaceStatus() to succeed here? hello-sharing-vaapi-export runs correctly, so Export() looks to be returning valid frames.

The failure in clCreateFromVA_APIMediaSurfaceINTEL() is probably due to some mismatch between the flags used in the test app and the VPL runtime. If you would like to try building a debug build of the runtime to step through it, the code is at https://github.com/intel/vpl-gpu-rt

[EmonLu]

Found that clCreateFromVA_APIMediaSurfaceINTEL() fails when encodeParams.mfx.FrameInfo.FourCC is MFX_FOURCC_BGR4, and successes for MFX_FOURCC_RGB4.

The exported surface from VPL is valid. Sorry about the mistake before that VAStatus = 4 actually means VASurfaceReady rather than invalid.

The function createSharedVaSurface in compute-runtime/opencl/source/sharings/va/va_surface.cpp returns VA_STATUS_ERROR_INVALID_PARAMETER due to it only supports FourCC below:

bool VASurface::isSupportedPackedFormat(uint32_t imageFourcc) {
    switch (imageFourcc) {
    case VA_FOURCC_YUY2:
        return true;
    case VA_FOURCC_Y210:
        return true;
    case VA_FOURCC_ARGB:
        return true;
    default:
        return false;
    }
}

How can I enable clCreateFromVA_APIMediaSurfaceINTEL() when encodeParams.mfx.FrameInfo.FourCC is MFX_FOURCC_BGR4?