Make User-space memory DMA'ble

[Miouyouyou]

The VPU basically does a DMA operation to read the frame source and writes the desired output.

Since these frames will come from a video player invoked by the user, in the first place, we need to :

• Make the memory buffer usable for DMA operations and/or allocate a DMA'ble memory and store the frame into it.
• Read from that buffer, while taking into account that the IOMMU has to be prepared by the CPU to perform the read through DMA operations.

Now, let's take a simple example : Decoding a frame using FFMPEG. The mechanism will be :

• [ FFMPEG : User memory : Input ] → [ Driver : Make a DMA'ble buffer from it ]
• [ FFMPEG : User memory : Output ] → [ Driver : Make DMA'ble buffer from it ]
• [ Driver : Prepare the VPU and setup the Input/Output buffers ] → [ Driver : Do the VPU transcoding ]
• [ FFMPEG : Read the output and write it to the disk ]

[Miouyouyou]

Allocating 64M of memory through DMA operations failed...

[26591.354557] test_dma_to_from_user: loading out-of-tree module taints kernel.
[26591.362697] Use dma_alloc_coherent
[26591.366521] ------------[ cut here ]------------
[26591.371700] WARNING: CPU: 2 PID: 13111 at mm/page_alloc.c:4061 __alloc_pages_nodemask+0x174/0xcf4
[26591.381628] Modules linked in: test_dma_to_from_user(O+) bnep mali_kbase r8723bs(C) rockchip_rga dw_hdmi_cec v4l2_mem2mem videobuf2_dma_sg rk_crypto
[26591.396545] CPU: 2 PID: 13111 Comm: insmod Tainted: G         C O      4.17.0-rc6-RockMyy-Seventeen #1
[26591.406944] Hardware name: Rockchip (Device Tree)
[26591.412203] [<c010fdec>] (unwind_backtrace) from [<c010bf88>] (show_stack+0x10/0x14)
[26591.420849] [<c010bf88>] (show_stack) from [<c0c93df0>] (dump_stack+0x70/0x8c)
[26591.428921] [<c0c93df0>] (dump_stack) from [<c011f930>] (__warn+0xd4/0xf0)
[26591.436602] [<c011f930>] (__warn) from [<c011fa0c>] (warn_slowpath_null+0x38/0x44)
[26591.445060] [<c011fa0c>] (warn_slowpath_null) from [<c01e9644>] (__alloc_pages_nodemask+0x174/0xcf4)
[26591.455269] [<c01e9644>] (__alloc_pages_nodemask) from [<c0116270>] (__dma_alloc_buffer.constprop.12+0x30/0x84)
[26591.466544] [<c0116270>] (__dma_alloc_buffer.constprop.12) from [<c01166b0>] (remap_allocator_alloc+0x24/0x74)
[26591.477721] [<c01166b0>] (remap_allocator_alloc) from [<c01146bc>] (__dma_alloc+0x1b4/0x278)
[26591.487148] [<c01146bc>] (__dma_alloc) from [<c01147e8>] (arm_dma_alloc+0x34/0x40)
[26591.495611] [<c01147e8>] (arm_dma_alloc) from [<bf16f070>] (test_user_dma_init+0x70/0x1000 [test_dma_to_from_user])
[26591.507280] [<bf16f070>] (test_user_dma_init [test_dma_to_from_user]) from [<c0102da0>] (do_one_initcall+0xd8/0x228)
[26591.519032] [<c0102da0>] (do_one_initcall) from [<c01981c8>] (do_init_module+0x58/0x1c8)
[26591.528073] [<c01981c8>] (do_init_module) from [<c019773c>] (load_module+0x1eb4/0x1f10)
[26591.537015] [<c019773c>] (load_module) from [<c0197968>] (sys_finit_module+0x88/0x90)
[26591.545763] [<c0197968>] (sys_finit_module) from [<c0101000>] (ret_fast_syscall+0x0/0x54)
[26591.554898] Exception stack(0xe4547fa8 to 0xe4547ff0)
[26591.560537] 7fa0:                   d55a6300 00000000 00000003 0043884c 00000000 be9ee670
[26591.569666] 7fc0: d55a6300 00000000 0044b000 0000017b 00000000 00000000 0044a9bc 00000000
[26591.578800] 7fe0: be9ee618 be9ee608 004304bb b6efd712
[26591.584491] ---[ end trace 86cfd03abe6d9429 ]---
[26591.589656] DMA Alloc coherent could not allocate
               Calling the cops right now !

I'll try to rewrite the test so that it sticks to the VPU device and see if playing with the IOMMU allows dma_alloc_coherent to use the DMA IOMMU framework "alloc" function instead.

[Miouyouyou]

Lowering the amount of memory allocated resolve the issue.

I'll try to reserve 64M of RAM with the CMA allocator and see how it goes.

[Miouyouyou]

Ample verifications must be done to ensure that the buffers allocated are zero'ed and that user <-> kernel sharing works correctly through a few simple IOCTL ping-pong checks.