Open jofemodo opened 2 months ago
6by9
commented
2 months ago If an older Pi5 bootloader works, then @timg236 for some weird bootloader change.
6by9
commented
2 months ago We have a hunch that it's related to the clone of the Pi display not handling reading other registers, which is totally valid on our display.
Simplest test is to add display_auto_detect=0 and dtoverlay=vc4-kms-dsi-7inch to config.txt, and confirming it still works (use dtoverlay=vc4-kms-dsi-7inch,dsi0 if using CAM/DISP0).
Running i2ctransfer -y -f 6 w1@0x45 0x80 i2ctransfer -y -f 6 r1@0x45 should return 0xc3.
If you can then try i2ctransfer -y -f 6 w1@0x45 0x01 i2ctransfer -y -f 6 r1@0x45 I suspect it will lock up the I2C bus.
I believe we have a workaround, but would like to confirm that it's going to work before deploying it. (If correct, then it is related to #5784 in that for both cases it is actually the peripheral going wrong!)
pelwell
commented
2 months ago Current kernels should be better at getting out of RP1 I2C error states, but there isn't much that can be done if CLK is being held low by a device.
jofemodo
commented
2 months ago I insist in that it seems firmware-related. It's totally consistent with the firmware version used.
I don't know if the device is behaving badly or not regarding the I2C buses, but for sure this display works without issues in RPi4 (more than 200 x V5 units and nobody reported issues), and it has been working in RPi5 for months, until a recent firmware update broke things. By restoring the old firmware (but keeping the latest kernel!!!) the display works perfectly OK.
Anyway, let me do the tests you suggest with i2ctransfer and so. I want to give you all the details you need.
Regards,
6by9
commented
2 months ago Yes the firmware has changed to read one additional register from I2C address 0x45. This is a legitimate thing to do on a Pi display, however we believe your clone display doesn't like it and locks up the bus.
You can't modify the firmware, so my steps above are to confirm that it is this additional read that is causing your panel to lock up. Once confirmed then I believe we can work around it.
jofemodo
commented
2 months ago Understood! Thanks for clarifying the issue. I suppose that adding a "flag" to the firmware, so it can behave as it used to do, is not too hard, and there are tons of "generic" DSI displays in the world that would need this to keep working with RPi5.
pelwell
commented
2 months ago They're not "generic" displays, they are intentionally made to look like one of our displays - clones, as 6by9 said.
jofemodo
commented
2 months ago Not mine. It doesn't look as a Pi display, simply try to "clone" or "implement" the Raspberry Pi DSI interface. It's called "5" TFT MODULE MIPI DSI". My provider doesn't sell the display as a "Pi Display", and of course i never thought it was a "Pi Display".
pelwell
commented
2 months ago For "look like" read "behave like" - I wasn't claiming it was a fake.
6by9
commented
2 months ago This is believed to be related to the autodetection of the displays.
Auto detect is there for genuine Raspberry Pi devices, but as these 800x480 panels pretend to the Pi to be the Pi 7" panel, the configuration for the Pi panel is loaded.
There already is the option to manually configure these by setting display_auto_detect=0 and specifying the relevant dtoverlay= line. We don't want to force users to do that, so need to confirm we understand what is going wrong.
Most I2C devices work by having addressed registers, so eg writing 0x80 0x12 as one I2C transaction to the relevant I2C address would set register 0x80 to value 0x12. Normally you can then read that back by writing 0x80 and then doing a 1 byte read.
The 7" DSI panel has always been detected by reading register 0x80. (That is the key bit that these displays have cloned, and then they have to be prepared to accept the same video signal that is configured for the Pi display).
For "reasons" we are now also reading register 0x01 during the auto detect. Any normal device would return some value to that read request, and indeed you'd reject as faulty any normal I2C device that misbehaved if you read a register that doesn't exist. I2C has the option for NAKing messages for example, and locking the bus is verboten.
However most of these panels use microcontrollers (MCUs) to implement the I2C peripheral, and there it is down to how the MCU firmware has been written as to what it does under any condition. Mess that firmware up and you can lock up the I2C bus - that is what we believe is happening here. If confirmed then we believe we can work around it.
jofemodo
commented
2 months ago For "look like" read "behave like" - I wasn't claiming it was a fake.
But this is legitimate. They tried to make them compatible with the RPi DSI interface, and i suppose they did their best, but probably they failed because of different reasons, but not willfully. Perhaps these details were not clear enough in the DSI technical documentation (BTW, i couldn't find it!), perhaps they didn't understand or perhaps they were simply lazy ... ;-)
jofemodo
commented
2 months ago I forgot to say that before opening this issue, i already tried to disable autodetection (display_auto_detect=0) and loading different overlays like these:
When i did, then i can launch X11 and the display seems to work:
# kmsprint
Connector 0 (35) DSI-2 (connected)
Encoder 0 (34) NONE
Crtc 0 (33) 840x480@62.38 30.000 840/46/2/44/? 480/16/2/18/? 62 (62.38)
Plane 0 (31) fb-id: 38 (crtcs: 0) 0,0 840x480 -> 0,0 840x480 (XR24 XB24 AR24 AB24 RG24 BG24 RG16)
FB 38 840x480 XR24
But nothing is displayed in the LCD. I tried to toggle the backlight, but no luck I also tried to read events from the touch device, but it's silent.
Thanks
6by9
commented
2 months ago I forgot to say that before opening this issue, i already tried to disable autodetection (display_auto_detect=0) and loading different overlays like these:
* vc4-kms-dsi-generic.dtbo * vc4-kms-dsi-7inch.dtbo * vc4-kms-dsi-waveshare-panel.dtbo
It's not a generic display (for which the overlay needs configuring) and it's not a Waveshare panel (for which the overlay needs telling which display variant it is). Throwing in random garbage around rarely works.
When i did, then i can launch X11 and the display seems to work:
But nothing is displayed in the LCD.
So the display works but nothing is displayed?!?!
I wanted to know about the I2C status and whether you can get it to lock up using the i2ctransfer commands.
I have made a test bootloader that should work around the issue as we understand it. Please:
display_auto_detect=1 and dtoverlay=vc4-kms-v3d.sudo systemctl disable rpi-eeprom-updatejofemodo
commented
2 months ago Ups!! I missed the "dsi0" parameter! My fault! Now it's working with this config:
display_auto_detect=0
dtoverlay=vc4-kms-dsi-7inch,dsi0For me, autodetection is not high priority so from my POV, the issue is "kind of solved". Perhaps you want to close or keep it open until you make the autodetection to work again.
Thanks a lot for sharing your knowledge and give me support so fast!! Really!!
jofemodo
commented
2 months ago Anyway, tell me if you still want extended info about I2C status using the i2ctransfer commands. I will do my best to help you to fully solve the issue.
6by9
commented
2 months ago For "look like" read "behave like" - I wasn't claiming it was a fake.
But this is legitimate. They tried to make them compatible with the RPi DSI interface, and i suppose they did their best, but probably they failed because of different reasons, but not willfully. Perhaps these details were not clear enough in the DSI technical documentation (BTW, i couldn't find it!), perhaps they didn't understand or perhaps they were simply lazy ... ;-)
If you buy a third party exhaust for your car that looks the same as the original. Do you complain to the car manufacturer if something inside of the exhaust causes the car to go wrong?
The Pi display is not open hardware, so they have reverse engineered it to look like the Pi display as they see it at the time.
We haven't deliberately gone out of our way to break it, but due to what most engineers would consider quite a serious error in their MCU firmware, it appears to have a way to fairly seriously screw the system up. We're happy to try and work around it (as above), but understanding an issue is key to ensuring the fix is correct, and means we hopefully avoid doing it again.
6by9
commented
2 months ago Anyway, tell me if you still want extended info about I2C status using the i2ctransfer commands. I will do my best to help you to fully solve the issue.
Yes, I would like you to confirm whether you can lock up their display controller, and tell me if the above test bootloader works.
jofemodo
commented
2 months ago If you buy a third party exhaust for your car that looks the same as the original. Do you complain to the car manufacturer if something inside of the exhaust causes the car to go wrong?
I understand and you are right, specially regarding the manufacturer lazyness when implementing the i2c firmware. They should have been made a more robust implementation, more I2C-complaint, but we all know "how relaxed" can be I2C interfaces of many chips and small devices, not only chinese ones ;-)
Anyway, i personally dislike the foundation's policy about the DSI interface. It's not just a question of money and nobody is asking you to release the Pi-Display schematics/PCBs, but you could open the DSI specification and allow manufacturers to be compliant instead of having to reverse engineering the interface. IMHO, closing a hardware interface and using this to monopolize a market segment is not very nice, specially when you are the Pi Foundation and "spreads" the word of open source. But the more important thing is that this harms the ecosystem by imposing limitations and creating troubles to legitimate players that want to make it grow.
(of course this is just MHO and i don't want to give moral lessons or so ... please excuse my "big mouth" ;-))
For instance, in my case, we needed an integrated display solution for our zynthian V5 design. Before walking the DSI path, we prototyped a custom HDMI display with a specific 24-pin FPC connector, that included HDMI and I2C for the touch. This allowed to make a integrated design, slim-enough, but it was suboptimal and resulted too expensive and hard to maintain. HDMI is not thought for this! So we decided to go DSI because this interface offered just what we needed (quite obvious! DSI is designed for integration). The only problem is that RBPi's DSI is not open. But we needed customized DSI displays and i doubt the RPi foundation is going to manufacture 500 custom 5inch Pi-display units for us (i must admit we don't even tried), so we walked the "clonic DSI" way because of the increasing availability of these generic DSI displays, and of course, because of the affordable price. We tested a few units and we didn't detect any issue so we were confident enough to go forward and reached a deal with a chinese manufacturer that customized 500 units that fitted our V5 design. That's all and V5 has been a real success!! What a beautiful device!! ;-)
All the best.
6by9
commented
2 months ago DSI has been fully open since Bullseye came out over 3 years ago, and was an option even before that if you selected vc4-kms-v3d manually. We're open to PRs adding 3rd party overlays hence the vc4-kms-dsi-* and cutie-pi overlays. Just browse Waveshare's range of DSI panels for the Pi. Driver patches are acceptable too within reason, although we'd prefer them to go upstream.
It's the 3rd party clones that piggy-back on our panel's configuration that causes these problems as we can't predict how they'll (mis)behave.
You do have to manually edit config.txt to add the relevant dtoverlay line, but that's largely as there is no one generic way to autodetect a DSI panel, and there needs to be a line drawn as to how complex the firmware gets for these things. (This issue would be a prime example - what would you do if 3rd party panel maker A asked us to autodetect by doing X, and that broke 3rd party panel maker B's display?)
Enough debate and discussion - please test the bootloader and report back.
jofemodo
commented
2 months ago
# pinctrl 38-41
38: a3 pu | hi // CD0_SDA/GPIO38 = SDA6
39: a3 pu | hi // CD0_SCL/GPIO39 = SCL6
40: no pd | hi // CD1_SDA/GPIO40 = none
41: no pd | hi // CD1_SCL/GPIO41 = none
root@zynthian:~# i2ctransfer -y -f 6 w1@0x45 0x80
root@zynthian:~# i2ctransfer -y -f 6 r1@0x45
Error: Sending messages failed: Connection timed out
root@zynthian:~# i2ctransfer -y -f 6 w1@0x45 0x80
Error: Sending messages failed: Connection timed out
root@zynthian:~# i2ctransfer -y -f 6 w1@0x45 0x80
Error: Sending messages failed: Connection timed out
root@zynthian:~# i2ctransfer -y -f 6 r1@0x45
Error: Sending messages failed: Connection timed out
root@zynthian:~# i2ctransfer -y -f 6 w1@0x45 0x80
Error: Sending messages failed: Connection timed out
root@zynthian:~# pinctrl 38-41
38: a3 pu | hi // CD0_SDA/GPIO38 = SDA6
39: a3 pu | lo // CD0_SCL/GPIO39 = SCL6
40: no pd | hi // CD1_SDA/GPIO40 = none
41: no pd | hi // CD1_SCL/GPIO41 = none
Please, note that the first transfer ran OK and all the rest failed. The I2C bus seems locked. I had to power-off / power-on to restore the display to work.
jofemodo
commented
2 months ago I repeated the test with the register 0x1. Same result.
jofemodo
commented
2 months ago I apologize for my comment above. It's clear that i was badly informed about the DSI and deserve some hard punishment to my silly words.
jofemodo
commented
2 months ago I just tested the firmware you attached and autodetection works again. Is it safe to use for production?
Thanks!
6by9
commented
2 months ago I just tested the firmware you attached and autodetection works again. Is it safe to use for production?
It's safe, but please don't. We will get this merged and released through apt as quickly as we can.
jofemodo
commented
2 months ago Not in a hurry ;-)
6by9
commented
1 month ago I've realised I made an error within the first test firmware, which would have issues under certain conditions (not yours). Revised version attached if you would care to test it. Same procedure as before.
6by9
commented
1 month ago Now released via rpi-update.
You may need to re-enable with sudo systemctl enable rpi-eeprom-update if using this on the system you previously put the test firmware on.
jofemodo
commented
1 month ago Big thanks @6by9!
Do we have an aprox. date for having this in the mainstream firmware?
Regards
6by9
commented
1 month ago We have started prepping a full image update, so I would expect it to get merged to apt in the next few weeks.
Describe the bug
After an update, my generic DSI 5 inch display stopped working. It's a generic display that i've been using with more than 200 devices with RPi4 + display (zynthian V5) without known issues. It has been working with the RPi5 for months, but suddenly, after an update, it stopped to work.
The issue seems to be firmware-related and also it seems related to this thread and others. I found this lines in the logs:
I did:
It seems that the I2C lines behave as described in #5784 and others, preventing the touch controller to be initialized, what seems to break the DSI overlay.
After reflashing an older firmware, the issue vanishes and the display works normally. Then i do:
Steps to reproduce the behaviour
I can reproduce the issue consistently by simply reflashing the recent firmware. In the other hand, by flashing an older firmware, the issue gets "fixed".
Device (s)
Raspberry Pi 5
System
I'm using latest RaspberryPiOS Bookworm aarch64, updated to the last official kernel and packages. No rpi-updates.
Logs
This is the log whe booting with the recent firmware, that causes the error:
After recovering an old firmware version, it works normally:
Additional context
Please, note that it's the same SD image with the same kernel and configuration. The only difference between the 2 boots is the firmware.
Also note that the display works flawlessly with RPi4 and it has been working with RPi5 for months.
I've "patched" the system with:
$ sudo systemctl mask rpi-eeprom-updatebut this is highly sub-optimal!