search

raspberrypi / linux

Kernel source tree for Raspberry Pi-provided kernel builds. Issues unrelated to the linux kernel should be posted on the community forum at https://forums.raspberrypi.com/
Other
10.98k stars 4.94k forks source link

Raspberry PI 5 issue with TV Hat #5953

Closed clementmoine closed 6 months ago

clementmoine commented 6 months ago

Describe the bug

Several Raspberry Pi 5 users with the TV Hat have reported issues, such as some kind of kernel panics and crashes, when the TV Hat is installed on the Pi 5 and when tvheadend or any other program attempts to access /dev/dvb. I am one of them; when I plug in the module, the Pi starts up normally, but as soon as tvheadend is launched, the Pi freezes and becomes inaccessible.

A thread is open on the raspberry pi forum but we can't get any help from there. I can't assure that issue come from the firmware but dmseg logs seems to confirm that.

I tried moving the SDCard from RPI 5 to RPI 4, everything works as expected. Turning back to the RPI 5 get the PI frozen after seconds.

https://forums.raspberrypi.com/viewtopic.php?t=359876

Steps to reproduce the behaviour

Device (s)

Raspberry Pi 5

System

https://pastebin.com/raw/tEmhhAAK

Logs

[ 34.145476] cxd2880 spi0.0: DVB: adapter 0 frontend 0 frequency 0 out of range (174000000..862000000) [ 53.040366] cxd2880: cxd2880_set_frontend: sys:3 freq:538000000 bw:8 [ 54.060973] cxd2880: cxd2880_set_frontend: tune result 0 [ 90.453494] spi_master spi0: RX FIFO overflow detected [ 90.466429] dma dma2chan1: dma2chan1 failed to stop [ 90.467499] cxd2880_spi: cxd2880_spi_read_ts: spi_sync failed [ 90.478839] spi_master spi0: RX FIFO overflow detected [ 90.478840] dma dma2chan1: dma2chan1 is non-idle! [ 90.479888] cxd2880_spi: cxd2880_spi_read_ts: spi_sync failed [ 90.632101] spi_master spi0: RX FIFO overflow detected [ 90.642251] dma dma2chan1: dma2chan1 failed to stop [ 90.647226] cxd2880_spi: cxd2880_spi_read_ts: spi_sync failed [ 90.647448] spi_master spi0: RX FIFO overflow detected [ 90.647451] dma dma2chan1: dma2chan1 is non-idle! [ 90.648495] cxd2880_spi: cxd2880_spi_read_ts: spi_sync failed

Additional context

No response

popcornmix commented 6 months ago

This isn't something I see. I use the TV HAT at home as a tvheadend backend. I'm using kernel=kernel8.img.

pi@dvbdonkey:~ $ uptime
 11:37:49 up 26 days, 14:33,  2 users,  load average: 0.02, 0.01, 0.00
pi@dvbdonkey:~ $ cat /proc/device-tree/compatible 
raspberrypi,5-model-bbrcm,bcm2712
pi@dvbdonkey:~ $ uname -a
Linux dvbdonkey 6.1.0-rpi7-rpi-v8 #1 SMP PREEMPT Debian 1:6.1.63-1+rpt1 (2023-11-24) aarch64 GNU/Linux
pi@dvbdonkey:~ $ systemctl status tvheadend.service 
● tvheadend.service - Tvheadend - a TV streaming server and DVR
     Loaded: loaded (/lib/systemd/system/tvheadend.service; enabled; preset: enabled)
     Active: active (running) since Fri 2024-01-26 21:04:10 GMT; 3 weeks 5 days ago
    Process: 825 ExecStart=/usr/bin/tvheadend -f -p /run/tvheadend.pid $OPTIONS (code=exited, status=0/SUCCESS)
   Main PID: 849 (tvheadend)
      Tasks: 32 (limit: 9133)
        CPU: 2h 22min 53.649s
     CGroup: /system.slice/tvheadend.service
             └─849 /usr/bin/tvheadend -f -p /run/tvheadend.pid -u hts -g video

Feb 22 02:54:29 dvbdonkey tvheadend[849]: epggrab: UK: Freeview - data completion timeout for 594MHz in DVB-T Network
Feb 22 02:54:29 dvbdonkey tvheadend[849]: subscription: 0C89: "epggrab" unsubscribing
Feb 22 02:54:30 dvbdonkey tvheadend[849]: mpegts: 586MHz in DVB-T Network - tuning on Sony CXD2880 #0 : DVB-T #0
Feb 22 02:54:30 dvbdonkey tvheadend[849]: subscription: 0C8B: "epggrab" subscribing to mux "586MHz", weight: 4, adapter: "Sony CXD2880 #0 : DVB-T #0", networ>
Feb 22 03:04:35 dvbdonkey tvheadend[849]: epggrab: UK: Freeview - data completion timeout for 586MHz in DVB-T Network
Feb 22 03:04:35 dvbdonkey tvheadend[849]: subscription: 0C8B: "epggrab" unsubscribing
Feb 22 03:04:36 dvbdonkey tvheadend[849]: mpegts: 570MHz in DVB-T Network - tuning on Sony CXD2880 #0 : DVB-T #0
Feb 22 03:04:36 dvbdonkey tvheadend[849]: subscription: 0C8D: "epggrab" subscribing to mux "570MHz", weight: 4, adapter: "Sony CXD2880 #0 : DVB-T #0", networ>
Feb 22 03:14:41 dvbdonkey tvheadend[849]: epggrab: EIT: EPG Grabber - data completion timeout for 570MHz in DVB-T Network
Feb 22 03:14:41 dvbdonkey tvheadend[849]: subscription: 0C8D: "epggrab" unsubscribing
pi@dvbdonkey:~ $ dmesg |tail
[2265558.496310] cxd2880: cxd2880_set_frontend: sys:3 freq:498000000 bw:8
[2265559.521097] cxd2880: cxd2880_set_frontend: tune result 0
[2266164.494928] cxd2880: cxd2880_set_frontend: sys:16 freq:474167000 bw:8
[2266165.527410] cxd2880: cxd2880_set_frontend: tune result 0
[2266770.491822] cxd2880: cxd2880_set_frontend: sys:3 freq:594000000 bw:8
[2266771.524057] cxd2880: cxd2880_set_frontend: tune result 0
[2267376.485770] cxd2880: cxd2880_set_frontend: sys:3 freq:586000000 bw:8
[2267377.510529] cxd2880: cxd2880_set_frontend: tune result 0
[2267982.482124] cxd2880: cxd2880_set_frontend: sys:3 freq:570000000 bw:8
[2267983.507543] cxd2880: cxd2880_set_frontend: tune result 0
clementmoine commented 6 months ago

This isn't something I see. I use the TV HAT at home as a tvheadend backend. I'm using kernel=kernel8.img.


pi@dvbdonkey:~ $ uptime

 11:37:49 up 26 days, 14:33,  2 users,  load average: 0.02, 0.01, 0.00

pi@dvbdonkey:~ $ cat /proc/device-tree/compatible 

raspberrypi,5-model-bbrcm,bcm2712

pi@dvbdonkey:~ $ uname -a

Linux dvbdonkey 6.1.0-rpi7-rpi-v8 #1 SMP PREEMPT Debian 1:6.1.63-1+rpt1 (2023-11-24) aarch64 GNU/Linux

pi@dvbdonkey:~ $ systemctl status tvheadend.service 

● tvheadend.service - Tvheadend - a TV streaming server and DVR

     Loaded: loaded (/lib/systemd/system/tvheadend.service; enabled; preset: enabled)

     Active: active (running) since Fri 2024-01-26 21:04:10 GMT; 3 weeks 5 days ago

    Process: 825 ExecStart=/usr/bin/tvheadend -f -p /run/tvheadend.pid $OPTIONS (code=exited, status=0/SUCCESS)

   Main PID: 849 (tvheadend)

      Tasks: 32 (limit: 9133)

        CPU: 2h 22min 53.649s

     CGroup: /system.slice/tvheadend.service

             └─849 /usr/bin/tvheadend -f -p /run/tvheadend.pid -u hts -g video

Feb 22 02:54:29 dvbdonkey tvheadend[849]: epggrab: UK: Freeview - data completion timeout for 594MHz in DVB-T Network

Feb 22 02:54:29 dvbdonkey tvheadend[849]: subscription: 0C89: "epggrab" unsubscribing

Feb 22 02:54:30 dvbdonkey tvheadend[849]: mpegts: 586MHz in DVB-T Network - tuning on Sony CXD2880 #0 : DVB-T #0

Feb 22 02:54:30 dvbdonkey tvheadend[849]: subscription: 0C8B: "epggrab" subscribing to mux "586MHz", weight: 4, adapter: "Sony CXD2880 #0 : DVB-T #0", networ>

Feb 22 03:04:35 dvbdonkey tvheadend[849]: epggrab: UK: Freeview - data completion timeout for 586MHz in DVB-T Network

Feb 22 03:04:35 dvbdonkey tvheadend[849]: subscription: 0C8B: "epggrab" unsubscribing

Feb 22 03:04:36 dvbdonkey tvheadend[849]: mpegts: 570MHz in DVB-T Network - tuning on Sony CXD2880 #0 : DVB-T #0

Feb 22 03:04:36 dvbdonkey tvheadend[849]: subscription: 0C8D: "epggrab" subscribing to mux "570MHz", weight: 4, adapter: "Sony CXD2880 #0 : DVB-T #0", networ>

Feb 22 03:14:41 dvbdonkey tvheadend[849]: epggrab: EIT: EPG Grabber - data completion timeout for 570MHz in DVB-T Network

Feb 22 03:14:41 dvbdonkey tvheadend[849]: subscription: 0C8D: "epggrab" unsubscribing

pi@dvbdonkey:~ $ dmesg |tail

[2265558.496310] cxd2880: cxd2880_set_frontend: sys:3 freq:498000000 bw:8

[2265559.521097] cxd2880: cxd2880_set_frontend: tune result 0

[2266164.494928] cxd2880: cxd2880_set_frontend: sys:16 freq:474167000 bw:8

[2266165.527410] cxd2880: cxd2880_set_frontend: tune result 0

[2266770.491822] cxd2880: cxd2880_set_frontend: sys:3 freq:594000000 bw:8

[2266771.524057] cxd2880: cxd2880_set_frontend: tune result 0

[2267376.485770] cxd2880: cxd2880_set_frontend: sys:3 freq:586000000 bw:8

[2267377.510529] cxd2880: cxd2880_set_frontend: tune result 0

[2267982.482124] cxd2880: cxd2880_set_frontend: sys:3 freq:570000000 bw:8

[2267983.507543] cxd2880: cxd2880_set_frontend: tune result 0

So you changed the kernel to kernel 8 in the config.txt and everything runs fine one your end ? I tried that months ago and stream was working for just seconds before being stuck

clementmoine commented 6 months ago

@popcornmix I tried plugging back the TV Hat and booting with the kernel8.img. When I run the TVHeadEnd docker container I get the following errors in the dmesg and the Pi SSH is stuck. Light on the Pi is on green, no blinking, nothing

pi@raspberry:~ $ cat /proc/device-tree/compatible 
raspberrypi,5-model-bbrcm,bcm2712

pi@raspberry:~ $ uname -a
Linux raspberry 6.6.17-v8-16k+ #1735 SMP PREEMPT Wed Feb 21 14:45:17 GMT 2024 aarch64 GNU/Linux

pi@raspberry:~ $  dmesg |tail
[  105.233954]  invoke_syscall+0x4c/0x118
[  105.237712]  el0_svc_common.constprop.1+0x88/0xf8
[  105.242430]  do_el0_svc+0x24/0x38
[  105.245751]  el0_svc+0x48/0xe8
[  105.248811]  el0t_64_sync_handler+0xa0/0xc8
[  105.253005]  el0t_64_sync+0x190/0x198
[  105.256676] Code: 54001a00 b9402ae1 b9400880 11000400 (f8616aba) 
[  105.262791] ---[ end trace 0000000000000000 ]---
[  105.267421] Internal error: Oops: 0000000096000005 [#4] PREEMPT SMP
[  105.273715] Modules linked in: iptable_nat iptable_filter wireguard libchacha20poly1305 chacha_neon poly1305_neon ip6_udp_tunnel udp_tunnel libcurve25519_generic libchacha xt_nat xt_tcpudp veth xt_conntrack nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack_netlink nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xfrm_user xfrm_algo xt_addrtype nft_compat nf_tables nfnetlink br_netfilter bridge stp llc rfcomm snd_seq_dummy snd_hrtimer snd_seq snd_seq_device overlay algif_hash algif_skcipher af_alg bnep binfmt_misc cxd2880 vc4 cxd2880_spi snd_soc_hdmi_codec dvb_core drm_display_helper cec brcmfmac_wcc spidev brcmfmac hci_uart drm_dma_helper aes_ce_blk aes_ce_cipher drm_kms_helper brcmutil btbcm ghash_ce bluetooth gf128mul sha2_ce pisp_be rpivid_hevc(C) cfg80211 v4l2_mem2mem snd_soc_core sha256_arm64 ecdh_generic videobuf2_dma_contig sha1_ce ecc videobuf2_memops libaes videobuf2_v4l2 videodev snd_compress snd_pcm_dmaengine rfkill raspberrypi_hwmon snd_pcm videobuf2_common snd_timer v3d i2c_brcmstb snd spi_bcm2835 mc gpio_keys

Edit : Here is the full dmesg log from bootup to the freeze : https://gist.github.com/clementmoine/371b360ecc29809bbc373526f21750df

popcornmix commented 6 months ago

pi@raspberry:~ $ uname -a Linux raspberry 6.6.17-v8-16k+ #1735 SMP PREEMPT Wed Feb 21 14:45:17 GMT 2024 aarch64 GNU/Linux

You've not switched to the kernel8.img correctly. You are sill running kernel_2712.img.

I suspect you've not edited the correct config.txt file, or you don't have a kernel8.img in the boot directory.

clementmoine commented 6 months ago

pi@raspberry:~ $ uname -a Linux raspberry 6.6.17-v8-16k+ #1735 SMP PREEMPT Wed Feb 21 14:45:17 GMT 2024 aarch64 GNU/Linux

You've not switched to the kernel8.img correctly. You are sill running kernel_2712.img.

I suspect you've not edited the correct config.txt file, or you don't have a kernel8.img in the boot directory.

I have edited the /boot/firmware/config.txt and added the line at the end of the file

kernel=kernel8.img

I the /boot/firmware, I have the following files :

pi@raspberry:~ $ ls -alh /boot/firmware/ | grep .img
-rwxr-xr-x 1 root root 8.9M Feb 22 10:23 kernel_2712.img
-rwxr-xr-x 1 root root 8.9M Feb 22 10:23 kernel8.img
popcornmix commented 6 months ago

show output of mount | grep boot and post your complete config.txt

clementmoine commented 6 months ago

Sure @popcornmix !

pi@raspberry:~ $ mount | grep boot
/dev/mmcblk0p1 on /boot/firmware type vfat (rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro)

And here is the /boot/firmware/config.txt

# For more options and information see
# http://rptl.io/configtxt
# Some settings may impact device functionality. See link above for details

# Uncomment some or all of these to enable the optional hardware interfaces
#dtparam=i2c_arm=on
#dtparam=i2s=on
#dtparam=spi=on

# Enable audio (loads snd_bcm2835)
dtparam=audio=on

# Additional overlays and parameters are documented
# /boot/firmware/overlays/README

# Automatically load overlays for detected cameras
camera_auto_detect=1

# Automatically load overlays for detected DSI displays
display_auto_detect=1

# Automatically load initramfs files, if found
auto_initramfs=1

# Enable DRM VC4 V3D driver
dtoverlay=vc4-kms-v3d
max_framebuffers=2

# Don't have the firmware create an initial video= setting in cmdline.txt.
# Use the kernel's default instead.
disable_fw_kms_setup=1

# Run in 64-bit mode
arm_64bit=1

# Disable compensation for displays with overscan
disable_overscan=1

# Run as fast as firmware / board allows
arm_boost=1

[cm4]
# Enable host mode on the 2711 built-in XHCI USB controller.
# This line should be removed if the legacy DWC2 controller is required
# (e.g. for USB device mode) or if USB support is not required.
otg_mode=1

[all]

kernel=kernel8.img
popcornmix commented 6 months ago

Report:

md5sum /boot/firmware/kernel*
clementmoine commented 6 months ago

Report:

md5sum /boot/firmware/kernel*
pi@raspberry:~ $ md5sum /boot/firmware/kernel*
b1b24a946b6dcb8c6e0c88e782065a10  /boot/firmware/kernel_2712.img
82f818d1e48be8dda2ccab846b6d1f75  /boot/firmware/kernel8.img
popcornmix commented 6 months ago

I'm a bit stumped: With your config.txt and rpi-update kernel:

pi@pi5:~ $ uname -a
Linux pi5 6.6.17-v8+ #1735 SMP PREEMPT Wed Feb 21 14:39:52 GMT 2024 aarch64 GNU/Linux

If I remove kernel=kernel8.img I get:

pi@pi5d0:~ $ uname -a
Linux pi5 6.6.17-v8-16k+ #1735 SMP PREEMPT Wed Feb 21 14:45:17 GMT 2024 aarch64 GNU/Linux

I'd try:

sudo fsck -fy /dev/mmcblk0p1

to check boot filesystem.

Assuming you have another machine that you can use to edit sdcard, I'd be interested if renaming kernel_2712.img to kernel_2712.bad changes anything. It may do nothing, boot into expected kernel8.img or stop booting completely. The answer may be helpful.

clementmoine commented 6 months ago

I'm a bit stumped: With your config.txt and rpi-update kernel:

pi@pi5:~ $ uname -a
Linux pi5 6.6.17-v8+ #1735 SMP PREEMPT Wed Feb 21 14:39:52 GMT 2024 aarch64 GNU/Linux

If I remove kernel=kernel8.img I get:

pi@pi5d0:~ $ uname -a
Linux pi5 6.6.17-v8-16k+ #1735 SMP PREEMPT Wed Feb 21 14:45:17 GMT 2024 aarch64 GNU/Linux

I'd try:

sudo fsck -fy /dev/mmcblk0p1

to check boot filesystem.

Assuming you have another machine that you can use to edit sdcard, I'd be interested if renaming kernel_2712.img to kernel_2712.bad changes anything. It may do nothing, boot into expected kernel8.img or stop booting completely. The answer may be helpful.

I tried sudo fsck -fy /dev/mmcblk0p1 and this is the output 

pi@raspberry:~ $ sudo fsck -fy /dev/mmcblk0p1
fsck from util-linux 2.38.1
fsck.fat 4.2 (2021-01-31)
There are differences between boot sector and its backup.
This is mostly harmless. Differences: (offset:original/backup)
  65:01/00
  Not automatically fixing this.
Dirty bit is set. Fs was not properly unmounted and some data may be corrupt.
 Automatically removing dirty bit.

*** Filesystem was changed ***
Writing changes.
/dev/mmcblk0p1: 470 files, 38613/261116 clusters
clementmoine commented 6 months ago

I renamed the kernel_2712.img to kernel_2712.bad and reboot but when I reboot there is no changed in the uname -a :

Linux raspberry 6.6.17-v8+ #1735 SMP PREEMPT Wed Feb 21 14:39:52 GMT 2024 aarch64 GNU/Linux

popcornmix commented 6 months ago

That is a change. You reported earlier:

pi@raspberry:~ $ uname -a Linux raspberry 6.6.17-v8-16k+ #1735 SMP PREEMPT Wed Feb 21 14:45:17 GMT 2024 aarch64 GNU/Linux

Note the "-16k" is the thing that stops TV HAT from working. Does TV HAT now work?

clementmoine commented 6 months ago

That is a change. You reported earlier:

pi@raspberry:~ $ uname -a Linux raspberry 6.6.17-v8-16k+ #1735 SMP PREEMPT Wed Feb 21 14:45:17 GMT 2024 aarch64 GNU/Linux

Note the "-16k" is the thing that stops TV HAT from working. Does TV HAT now work?

Ok ok ! I tried plug the TV Hat and reboot, no crash but .... here is the logs of the dmesg :

[   50.319694] cxd2880 spi0.0: DVB: adapter 0 frontend 0 frequency 0 out of range (174000000..862000000)
[   57.502771] warning: `ThreadPoolForeg' uses wireless extensions which will stop working for Wi-Fi 7 hardware; use nl80211
[   77.069343] cxd2880: cxd2880_set_frontend: sys:3 freq:538000000 bw:8
[   78.090633] cxd2880: cxd2880_set_frontend: tune result 0
[   96.331782] cxd2880: cxd2880_set_frontend: sys:3 freq:490000000 bw:8
[   97.357536] cxd2880: cxd2880_set_frontend: tune result 0
[   98.044500] spi_master spi0: RX FIFO overflow detected
[   98.094807] dma dma2chan1: dma2chan1 failed to stop
[   98.095845] cxd2880_spi: cxd2880_spi_read_ts: spi_sync failed
[   98.096050] dma dma2chan1: dma2chan1 is non-idle!
[   98.096054] spi_master spi0: RX FIFO overflow detected
[   98.096086] cxd2880_spi: cxd2880_spi_read_ts: spi_sync failed
[   98.195620] cxd2880: cxd2880_read_status: failed. sys = 1
[  116.638108] cxd2880: cxd2880_set_frontend: sys:3 freq:490000000 bw:8
[  116.640278] cxd2880: cxd2880_set_frontend: tune result -22
[  116.640282] cxd2880: cxd2880_tune: cxd2880_set_frontend failed -22
[  124.612967] cxd2880: cxd2880_set_frontend: sys:3 freq:490000000 bw:8
[  124.614858] cxd2880: cxd2880_set_frontend: tune result -22
[  124.614862] cxd2880: cxd2880_tune: cxd2880_set_frontend failed -22
[  141.849005] cxd2880: cxd2880_set_frontend: sys:3 freq:538000000 bw:8
[  141.850881] cxd2880: cxd2880_set_frontend: tune result -22
[  141.850887] cxd2880: cxd2880_tune: cxd2880_set_frontend failed -22
[  147.849350] cxd2880: cxd2880_set_frontend: sys:3 freq:538000000 bw:8
[  147.850940] cxd2880: cxd2880_set_frontend: tune result -22
[  147.850945] cxd2880: cxd2880_tune: cxd2880_set_frontend failed -22

I tried running a channel in VLC but that was not showing anything so I am trying scanning again this may solve the VLC problem but the logs in dmesg are quite strange right ?

Edit :

Well no chance there, I tried running the scan again but there is no services found, I will try again on the PI 4 to see if I can find services from there

popcornmix commented 6 months ago

I don't get the overflow messages in my log.

My vague understanding of how dvb works if you tune into a multiplex which may contain several channels. The bandwidth for the whole multiplex is too great for spi, but you can typically get a few channels at once (e.g. I can watch one channel from a multiplex and record another). I wonder if the method you are using to access the streams is trying to grab the whole multiplex? (which can only fail).

If this is what is happening I'd expect this to fail on a Pi4 too.

I typically run tvheadend on the Pi5, and then play channels using kodi on a different Pi. For testing I also use TvhClient (on Iphone, but I think there is an Android option) which also works (choose play with internal VLC).

clementmoine commented 6 months ago

Well ! I rebooted the PI after check on the PI 4 and running everything fine (as expected) I plugged back everything on the PI5 and now everything is working ! Man I can't explain what was the issue ! Thank you a lot for your help !

Ntoesse96 commented 4 months ago

I have the same problem but I don't understand how you solved it, as soon as I installed TV Hat, the rasp 5 didn't work and I solved it using kernel=kernel8.img, but since I switched booting from micro SD to an NVMe M.2 SSD it gives me the same problem as you

popcornmix commented 4 months ago

What does uname -a report?

clementmoine commented 4 months ago

I have the same problem but I don't understand how you solved it, as soon as I installed TV Hat, the rasp 5 didn't work and I solved it using kernel=kernel8.img, but since I switched booting from micro SD to an NVMe M.2 SSD it gives me the same problem as you

I had the issue once again, to solve the problem, you need to remove or rename the kernel_2712.img to force the Pi to use the Kernel 8, in my case without renaming or removing the 2712 file Kernel 8 was never used.

When you use "uname -a" you should never see 16k mention