DRM_IOCTL_I915_GEM_USERPTR fails as regular user in "video" group

[jbeich]

lang/beignet wants to to use DRM_IOCTL_I915_GEM_USERPTR for OpenCL 2.0 support and CL_DEVICE_HOST_UNIFIED_MEMORY, see bug 217635. I'm not sure why but it doesn't work without root.

$ cat a.c
#include <err.h>
#include <errno.h>
#include <fcntl.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

#include <xf86drm.h>
#include <libdrm/i915_drm.h>

int main()
{
    int ret;
    long pgsz = sysconf(_SC_PAGESIZE);
    void *ptr = aligned_alloc(pgsz, pgsz);

    struct drm_i915_gem_userptr userptr;
    memset(&userptr, 0, sizeof(userptr));
    userptr.user_ptr = (__u64)(unsigned long)ptr;
    userptr.user_size = pgsz;

    int fd = open("/dev/dri/card0", O_RDWR, 0);

retry:
    ret = drmIoctl(fd, DRM_IOCTL_I915_GEM_USERPTR, &userptr);
    if (ret) {
    warn("DRM_IOCTL_I915_GEM_USERPTR");
    if (errno == ENODEV && userptr.flags == 0) {
        userptr.flags = I915_USERPTR_UNSYNCHRONIZED;
        goto retry;
    }
    }

    return ret;
}

$ pkg install libdrm pkgconf
$ cc a.c $(pkg-config --cflags --libs libdrm)

$ ls -lL /dev/dri/card0
crw-rw----  1 root  video  0x254 Mar 22 23:08 /dev/dri/card0
$ id
uid=1234(foo) gid=1234(foo) groups=1234(foo),0(wheel),44(video)

$ ./a.out
a.out: DRM_IOCTL_I915_GEM_USERPTR: Operation not supported by device
a.out: DRM_IOCTL_I915_GEM_USERPTR: Operation not permitted
zsh: exit 255

$ sudo ./a.out
a.out: DRM_IOCTL_I915_GEM_USERPTR: Operation not supported by device

[markjdb]

It's because we don't define CONFIG_MMU_NOTIFIER, so the stub definition of i915_gem_userptr_init__mmu_notifier() is used. That definition only allows unsynchronized userptr objects, and such objects can probably be used to trigger nasty system behaviour since there's nothing preventing the process from unmapping pages bound to a GEM userptr object.

[jbeich]

graphics/mesa-dri also uses this ioctl for anv (intel vulkan driver) but it's no longer required since 19.0.

[jbeich]

Even I915_USERPTR_UNSYNCHRONIZED is broken in drm-v4.11 or later. Affects SNA in xf86-video-intel, see https://github.com/FreeBSDDesktop/kms-drm/issues/32.

[jbeich]

lang/intel-compute-runtime also requires this ioctl. Sadly, I915_USERPTR_UNSYNCHRONIZED hack only works fine on drm-v4.9.

$ clinfo
ioctl(I915_GEM_USERPTR) failed. Try running as root but expect poor stability.
Abort was called at 59 line in file:
runtime/os_interface/linux/drm_memory_manager.cpp

$ sudo clinfo | head -5
Number of platforms                               1
  Platform Name                                   Intel(R) OpenCL HD Graphics
  Platform Vendor                                 Intel(R) Corporation
  Platform Version                                OpenCL 2.1
  Platform Profile                                FULL_PROFILE

$ sudo clpeak

Platform: Intel(R) OpenCL HD Graphics
  Device: Intel(R) Gen9 HD Graphics NEO
    Driver version  : 19.48.14977 (FreeBSD)
    Compute units   : 24
    Clock frequency : 0 MHz

    Global memory bandwidth (GBPS)
      float   : Abort was called at 91 line in file:
runtime/os_interface/linux/drm_command_stream.inl

[jbeich]

multimedia/libva-intel-media-driver also requires this ioctl for some QSV-only features. Sadly, I915_USERPTR_UNSYNCHRONIZED hack only works fine on drm-v4.9.

$ ffmpeg -hide_banner \
  -init_hw_device qsv=hw -filter_hw_device hw \
  -i elephants_dream_720p24.y4m \
  -vf hwupload=extra_hw_frames=64,format=qsv -c:v h264_qsv \
  -global_quality 18 -look_ahead 1 \
  -y elephants_dream_720p24.h264.mp4
Input #0, yuv4mpegpipe, from 'elephants_dream_720p24.y4m':
  Duration: 00:10:53.79, start: 0.000000, bitrate: 265421 kb/s
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p(progressive), 1280x720, SAR 1:1 DAR 16:9, 24 fps, 24 tbr, 24 tbn, 24 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_qsv))
Press [q] to stop, [?] for help
VPP PIPELINE:
RESIZE
VPP PIPELINE:
RESIZE
VPP PIPELINE:
RESIZE
VPP PIPELINE:
RESIZE
mos_bo_alloc_userptr:83
[CM]:   CRITICAL - HalCm_AllocateBuffer_Linux:554: Fail to Alloc BufferUP   58880 bytes (58880 x 1 BufferUP resource)

[CM]:   CRITICAL - AllocateBuffer:832: MOS return error [34]
[CM]:   CRITICAL - CreateBuffer:750: Error: Falied to allocate buffer.
CreateBufferUP:162: return check failed
Assertion failed: (!"CmRuntimeError"), function CmRuntimeError, file _studio/mfx_lib/encode_hw/h264/include/mfx_h264_encode_cm.h, line 65.

$ sudo ffmpeg ...
Input #0, yuv4mpegpipe, from 'elephants_dream_720p24.y4m':
  Duration: 00:10:53.79, start: 0.000000, bitrate: 265421 kb/s
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p(progressive), 1280x720, SAR 1:1 DAR 16:9, 24 fps, 24 tbr, 24 tbn, 24 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_qsv))
Press [q] to stop, [?] for help
VPP PIPELINE:
RESIZE
VPP PIPELINE:
RESIZE
VPP PIPELINE:
RESIZE
VPP PIPELINE:
RESIZE
Output #0, mp4, to 'elephants_dream_720p24.h264.mp4':
  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0: Video: h264 (h264_qsv) (avc1 / 0x31637661), qsv, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 1000 kb/s, 24 fps, 12288 tbn, 24 tbc
    Metadata:
      encoder         : Lavc58.54.100 h264_qsv
    Side data:
      cpb: bitrate max/min/avg: 0/0/1000000 buffer size: 0 vbv_delay: -1
[MOS]:  CRITICAL - Mos_Specific_GetMemoryCompressionMode:6026: Invalid (nullptr) Pointer.
[MOS]:  CRITICAL - SubmitCommandBuffer:1108: Command buffer submission failed!
[CM]:   CRITICAL - SubmitCommands:1160: hr check failed [34]
[CM]:   CRITICAL - HalCm_ExecuteTask:8400: MOS return error [34]
[CM]:   CRITICAL - FlushGeneralTask:2731: MOS return error [34]
Assertion failed: (!"CmRuntimeError"), function CmRuntimeError, file _studio/mfx_lib/encode_hw/h264/include/mfx_h264_encode_cm.h, line 65.

[valpackett]

We need CONFIG_MMU_NOTIFIER, which is currently notyet. (confusingly, normal grep would tell you that it's enabled, haha)

userptr requires mmu-notifier for full unprivileged support

https://patchwork.kernel.org/patch/6835101/

[valpackett]

*facepalm* that's already been mentioned in this thread back in 2017. Why did I spend time investigating :D

[valpackett]

I915_USERPTR_UNSYNCHRONIZED hack works fine for me on 5.0 though (on kaby lake) in ffmpeg qsv

[hselasky]

Is this fixed by: https://github.com/FreeBSDDesktop/kms-drm/pull/205

[jbeich]

@hselasky, no. Did you try the test? I915_USERPTR_UNSYNCHRONIZED only works as root and requires patching to use it as a fallback.

[CallumA]

This issue also appears to block lang/intel-compute-runtime running as root within a jail:

# clinfo
Abort was called at 56 line in file:
/wrkdirs/usr/ports/lang/intel-compute-runtime/work/compute-runtime-20.14.16441/shared/source/os_interface/linux/drm_memory_manager.cpp 

[jbeich]

After https://github.com/torvalds/linux/commit/c6bcc0c2fdfd (since drm-515-kmod) userptr is completely unusable on FreeBSD. Maybe a good excuse to remove intel-compute-runtime as rebasing patches was PITA, anyway. OpenCL on Intel GPUs can still be used via Rusticl in mesa-devel. However, there would be nothing to provide runtime for level-zero.