Enable VC4 with mainline kernel on RPi 4

[starnight]

According to #43, I make a blind shot and try to enable VC4 on RPi 4. Tested with latest Linux mainline kernel (5.5-rc3). Got the idea from from arch/arm/boot/dts/bcm2835-common.dtsi. I add the gpu node into bcm2711.dtsi

diff --git a/arch/arm/boot/dts/bcm2711.dtsi b/arch/arm/boot/dts/bcm2711.dtsi
index e2f6ffb00aa9..bfcb9ab5673b 100644
--- a/arch/arm/boot/dts/bcm2711.dtsi
+++ b/arch/arm/boot/dts/bcm2711.dtsi
@@ -263,6 +263,10 @@
        hvs@7e400000 {
            interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH>;
        };
+
+       gpu {
+           compatible = "brcm,bcm2835-vc4";
+       };
    };

    arm-pmu {

Then, The vc4 module is loaded.

$ lsmod | grep -E "(drm|vc4)"
vc4                   184320  1
cec                    57344  1 vc4
drm_kms_helper        225280  2 vc4
syscopyarea            16384  1 drm_kms_helper
sysfillrect            16384  1 drm_kms_helper
sysimgblt              16384  1 drm_kms_helper
fb_sys_fops            16384  1 drm_kms_helper
drm                   589824  4 drm_kms_helper,vc4

The frame buffer swtiches to vc4drmfb from simple. But, it cannot find any crtc or sizes.

$ dmesg | grep -E "(vc4|drm|modeset)"
[   21.900360] vc4-drm soc:gpu: bound fe806000.vec (ops vc4_vec_ops [vc4])
[   21.907331] vc4-drm soc:gpu: bound fe004000.txp (ops vc4_txp_ops [vc4])
[   21.920922] vc4-drm soc:gpu: bound fe400000.hvs (ops vc4_hvs_ops [vc4])
[   21.927741] fb0: switching to vc4drmfb from simple
[   21.938929] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[   21.945682] [drm] Driver supports precise vblank timestamp query.
[   21.972483] [drm] Initialized vc4 0.0.0 20140616 for soc:gpu on minor 0
[   21.982639] [drm] Cannot find any crtc or sizes
[   23.604477] broken atomic modeset userspace detected, disabling atomic
[   24.061635] broken atomic modeset userspace detected, disabling atomic

The full dmesg is as the attachment. dmesg.log

The GPU is listed under /dev/dri/

$ ls -l /dev/dri/*
crw-rw---- 1 root video 226, 0 Dec 26 14:13 /dev/dri/card0

/dev/dri/by-path:
total 0
lrwxrwxrwx 1 root root 8 Dec 26 14:13 platform-soc:gpu-card -> ../card0

Any suggestion to make it go further will be appreciated.

[nullr0ute]

The hardware on the RPi4/bcm2711 uses a newer version of VideoCore which has a different driver in the linux kernel called v3d so I wouldn't expect the vc4 driver to work.

[lategoodbye]

Sorry, i don't have any idea how to enable framebuffer on Raspberry Pi 4.

Maybe @itoral or @naushir from Igalia can shed some light on this.

Edit: This was the last relevant discussion i was involved.

[lategoodbye]

Just a note: AFAIK the vc4 kms driver is still required in Mainline for Raspberry Pi 4. But it's very likely that we also need adjustments in the low level drivers (power domain, clock).

[dsd]

Yes. Confusing as the naming may be, I believe the situation is:

RPi3 Kernel driver for modesetting: vc4 Kernel driver for 3D acceleraton: v3d Mesa driver for 3D acceleration: vc4

RPi4 Kernel driver for modesetting: vc4 Kernel driver for 3D acceleraton: v3d Mesa driver for 3D acceleration: v3d

This issue is about adding RPi4 support in the upstream kernel vc4 modesetting driver - the next logical step towards a full upstream graphics & display pipeline on RPi4.

On RPi3, the Raspbian vc4 setup lets you choose between full KMS (where the kernel drives the display controller directly), or firmware KMS (where the kernel asks the firmware to handle display controller interactions). However, the vc4 upstream kernel driver only supports full KMS on RPi3.

On RPi4, Raspbian currently only supports firmware KMS...

I presume there are reasons why firmware KMS was not upstreamed in the RPi3 days - either through upstream viewing it as conceptually inferior or unsuitable, or simply that it would require some rearchitecting before it would be accepted upstream, and nobody has stepped up.

I assume the logical path towards vc4 modesetting on RPi4 is (like RPi3) to implement full KMS support upstream, but unlike other aspects of RPi4 upstreaming, this can't be done by studying and rewriting existing Raspbian kernel code, since RPi4 full KMS is not currently implemented there.

If I'm on the right lines, it would be good if knowledeable people (Igalia?) could detail:

1. Are there any public resources/codebases/datasheets that can be used by interested parties to help develop an upstream implementation of full KMS support on RPi4?

2. Are there any current codebases, efforts or roadmaps around doing this work, which would be open for collaboration?

[itoral]

We (Igalia) are only involved with the Mesa driver and are not really involved with the display engine, so I am afraid we can't be of help here. I am not aware of any public resources describing the display pipeline on the RPI4 either, but I think @naushir would be the right person to confirm that.

[naushir]

@6by9 could you share some thoughts?

[6by9]

A Full KMS driver for Pi4 is being worked on by a subcontractor, and will be mainlined.

My early steps enabling DPI displays on Pi4 are at https://github.com/raspberrypi/linux/pull/3311. That has now been extended and has a single HDMI output running for basic stuff. Dual HDMI is very nearly there. There's a fair amount of tidying up and testing to do though.

FYI, Pi4 still has the same DPI, DSI, and VEC blocks. The HVS and HDMI blocks have significant updates, and the pixel valve routing has been changed. My changes mainly updated the HVS code, and relied on most of the rest not having changed. The I2C block connected to the HDMI ports is also different from previous versions, which is the most likely reason not to get any usable modes if you had tried enabling it.

[dsd]

That's excellent news. Thanks for including the mainlining aspect in the roadmap right from the start!

@starnight and I would be happy to help testing this new code when the time is right, as well as doing initial diagnosis of any issues that we encounter.

[dsd]

At https://github.com/raspberrypi/linux/issues/3391 @lategoodbye wrote:

Currently the mainline development of the RPi 4 requires a Serial-USB-Adapter. So at least framebuffer support for Linux & U-Boot would be a big improvement.

(I think my reply is further offtopic there, so I'll put it here instead.)

If all you want is an inflexible framebuffer, this is already working. Using a standard u-boot rpi4 build, at boot-time you can see the u-boot messages printed on the HDMI display. Presumably it is using a framebuffer set up by the RPi firmware.

Then I guess u-boot has some magic to insert the framebuffer address and dimensions into the device tree or something like that, because when we then boot a mainline kernel, the framebuffer continues working under Linux and even through to the X11/GNOME UI. There may be a special kernel config option needed for this to happen, but it's already active in our Ubuntu-based kernel build (using the ordinary Ubuntu kernel, not the RPi-specific one).

Here's an image with all this hooked up: Endless OS RPi4

[lategoodbye]

I prepared a branch based on Linux 5.7-rc1 and applied V2 of Maxime's VC4 KMS patches plus some minor adjustments: https://github.com/lategoodbye/rpi-zero/tree/v5.7-bcm2711-vc4

[dsd]

@6by9 thanks again for your support of mainling efforts here. Maxime's VC4 work seems to be developing nicely, @starnight showed me it running and will provide feedback upstream around the minor issues encountered.

With that fantastic progress there, I am naturally curious about the next step in the path to a full multimedia experience: do you have any plans/efforts concerning upstreaming of the V3D kernel-side GPU driver support for RPi4?

[6by9]

v3d was supported upstream even before Pi4 launched. Eric had done the majority of the work as VC6 V3D was supported on other set top box chips. Igalia have taken over (contracted to Raspberry Pi) now that Eric has moved on to pastures new, and they've been pushing fixes in both kernel and mesa.

Various drivers are being upstreamed for libcamera support by Ideas On Board (again contracted by Raspberry Pi). Currently that hasn't included the codecs driver, but that is a relatively simple bolt-on to the vchi, vcsm-cma, and ISP drivers that they are upstreaming.

[dsd]

Thanks for the fast response!

I see - I had not spotted that Linux's drivers/gpu/drm/v3d is a specific driver for this new generation GPU (and separate from the older gen driver at drivers/gpu/drm/vc4/vc4_v3d.c).

There's currently no applicable device node that binds to that driver in the upstream RPi4 device tree, but presumably glueing these pieces together is a fairly ordinary upstreaming task that could be done by the community as has been done for other bits.

[lategoodbye]

Now that the VC4 has been extended for BCM2711, i prepared a experimental branch. It bases on v5.10 and contains all the necessary changes to get V3D probing. Personally i tested only 32 bit ARM (multi_v7_defconfig). Most of the patches are cherry picked from downstream and are not ready for upstreaming yet.

[vianpl]

Now that the VC4 has been extended for BCM2711, i prepared a experimental branch. It bases on v5.10 and contains all the necessary changes to get V3D probing. Personally i tested only 32 bit ARM (multi_v7_defconfig). Most of the patches are cherry picked from downstream and are not ready for upstreaming yet.

@lategoodbye Funnily enough I just started looking a this myself some days ago (https://github.com/vianpl/linux/tree/v3d-dev). I made some progress with RPM, no more MMU corruptions. I still have to clean up the power domains patch, AFAIR it's hard to get right without internal design knowledge.

[lategoodbye]

Oops. It seems to be ideal time for kernel hacking. At least we could "merge" our efforts.

[lategoodbye]

Just some comments:

• strongly suggest to introduce a bcm2711 compatible for bcm2835-power
• not sure it's worth the effort but the V3D YAML should try to preserve the dependencies (register, interrupts), that's why i marked my version as WIP
• V3D is credulous about register values during probe, in case power domain is not enabled the registers will return 0xdeadbeef and the driver will crash

[newperson1746]

Tried to apply the patches to 5.11-rc2 for v3d (last few in v5.10-v3d branch), compiled for ARM64 and it all works except at boot:

v3d scb:gpu@7ec00000: invalid resource
v3d: probe of scb:gpu@7ec00000 failed with error -22

[newperson1746]

Just examined the differences between lategoodbye's patches (that I used) and vianpl's, and: https://github.com/vianpl/linux/commit/0147d8f5466d058f68f33951b447bd500f1a14f0 https://github.com/lategoodbye/rpi-zero/commit/aa0b600b6800cbff7f57166e703369f6e06208a7

vianpl, you put v3d in bcm2711.dtsi at 7ec04000 lategoodbye, you put v3d in bcm2711.dtsi at 7ec00000

Is the difference for a reason? I'm gonna try to change to vianpl's address and recompile. (i think the firmware-clock parameter is specified in a different location as well, but it shouldn't matter)

EDIT: looking at raspberry pi foundation downstream, in their custom dtsi for pi 4: https://github.com/raspberrypi/linux/blob/2598419a84c9dc71ee7fd534a9f68e91e7d701dc/arch/arm/boot/dts/bcm2711-rpi.dtsi Here they correct 7ec00000 to 7ec04000 only for the pi4...

here they delete it from the original larger dts (bcm270x-rpi.dtsi) /delete-node/ v3d@7ec00000; and change it to v3d: v3d@7ec04000 {

EDIT 2: still fails with same error just at a different address.

[lategoodbye]

Now the V3D driver has been accepted, i will close this.