Open seniorgod opened 3 years ago
Joulinar
commented
3 years ago Hi,
basically DietPi is not providing any kernel. This is done by the base image uses. In your case it is a Raspberry OS. Means you are running Raspberry OS Buster Kernel, provided by Raspberry Pi Foundation. And yes there is a totally different kernel used on Buster compare to Stretch.
Stretch: 4.19.42-v7+ Buster: 5.4.83-v7+
seniorgod
commented
3 years ago Hm,
i always thought that you have a „special" kernel supplied by allo. Isn’t it?
Am 02.02.2021 um 22:49 schrieb Joulinar notifications@github.com:
Hi,
basically DietPi is not providing any kernel. This is done by the base image uses. In your case it is a Raspberry OS. Means you are running Raspberry OS Buster Kernel, provided by Raspberry Pi Foundation. And yes there is a totally different kernel used on Buster compare to Stretch.
Stretch: 4.19.42-v7+ Buster: 5.4.83-v7+
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MichaIng/DietPi/issues/4080#issuecomment-772018404, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEC2TDWAVWTRYAKOUMYXLLLS5BXNVANCNFSM4W7SSPKQ.
Joulinar
commented
3 years ago I guess @MichaIng could explain it way better 😃
MichaIng
commented
3 years ago The kernel is the same indeed, the official Allo Sparky SBC kernel including patches. Could you give some details on the errors you face?
seniorgod
commented
3 years ago I needed to restart my sparky. Therefore i lost the kernel trace. As soon as i face the error again i try to capture the kernel trace.
Am 03.02.2021 um 01:09 schrieb MichaIng notifications@github.com:
The kernel is the same indeed, the official Allo Sparky SBC kernel https://github.com/sparkysbc/Linux including patches https://github.com/sparky-sbc/sparky-test. Could you give some details on the errors you face?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MichaIng/DietPi/issues/4080#issuecomment-772103321, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEC2TDX5EM7VOTFYGCDLAQTS5CH3JANCNFSM4W7SSPKQ.
seniorgod
commented
3 years ago Dear all, until now it the sparky don’t stop again.
But nevertheless i face kernel traces while playing music from time to time:
this warning occurred at startup:
----udc_set_plugstate--PLUGSTATE_OUT-- Feb 06 09:17:02 DietPi kernel: [PowerGate] name: 'usb3', on: 0, before SPS_PG_CTL: 0x717761 Feb 06 09:17:02 DietPi kernel: [PowerGate] name: 'usb3', on: 0, after SPS_PG_CTL: 0x717361 Feb 06 09:17:02 DietPi kernel: ------------dwc3_clk_exit---------- Feb 06 09:17:02 DietPi kernel: ++++++++cpufreq ready++++++++ Feb 06 09:17:02 DietPi kernel: ++++++++pdata->freq_tab[0].freq_clip_max:504000++++++++ Feb 06 09:17:02 DietPi kernel: ++++++++pdata->freq_tab[1].freq_clip_max:504000++++++++ Feb 06 09:17:02 DietPi kernel: ------------[ cut here ]------------ Feb 06 09:17:02 DietPi kernel: WARNING: at /imxhdd/opt/1301TAG/sparky_volumio/kernel/drivers/gpio/gpiolib.c:160 gpio_ensure_requested+0x58/0xb4()
Feb 06 09:17:02 DietPi kernel: autorequest GPIO-114
Feb 06 09:17:02 DietPi kernel: Modules linked in:
Feb 06 09:17:02 DietPi kernel: CPU: 0 PID: 218 Comm: kworker/0:3 Not tainted 3.10.38 #22
Feb 06 09:17:02 DietPi kernel: Workqueue: events wait_cpufreq_ready
Feb 06 09:17:02 DietPi kernel: [
later on i get that error:
atc260x-rtc atc2603c-rtc.0: atc260x_rtc_settime(): 2021-02-06 10:39:39
Feb 06 11:50:38 DietPi kernel: atc260x-rtc atc2603c-rtc.0: atc260x_rtc_settime(): 2021-02-06 10:50:39
Feb 06 11:54:13 DietPi kernel: ------------[ cut here ]------------
Feb 06 11:54:13 DietPi kernel: WARNING: at /imxhdd/opt/1301TAG/sparky_volumio/kernel/net/sched/sch_generic.c:255 dev_watchdog+0x24c/0x26c()
Feb 06 11:54:13 DietPi kernel: NETDEV WATCHDOG: eth0 (owl-ethernet): transmit queue 0 timed out
Feb 06 11:54:13 DietPi kernel: Modules linked in: nls_cp437 snd_usb_audio snd_hwdep snd_usbmidi_lib ethernet spidev atc260x_irkeypad atc260x_cap_ga
uge autofs4
Feb 06 11:54:13 DietPi kernel: CPU: 2 PID: 0 Comm: swapper/2 Tainted: G W 3.10.38 #22
Feb 06 11:54:13 DietPi kernel: [
and again:
Feb 06 21:25:57 DietPi kernel: ------------[ cut here ]------------
Feb 06 21:25:57 DietPi kernel: WARNING: at /imxhdd/opt/1301TAG/sparky_volumio/kernel/drivers/usb
/core/urb.c:328 usb_submit_urb+0x3d4/0x404()
Feb 06 21:25:57 DietPi kernel: URB e1ef9700 submitted while active
Feb 06 21:25:57 DietPi kernel: Modules linked in: nls_cp437 snd_usb_audio snd_hwdep sndusbmidi
lib ethernet spidev atc260x_irkeypad atc260x_cap_gauge autofs4
Feb 06 21:25:57 DietPi kernel: CPU: 0 PID: 1067 Comm: rtio Tainted: G W 3.10.38 #22
Feb 06 21:25:57 DietPi kernel: [
] (snd_pcm_common_ioctl1+0x828/0xd4c) Feb 06 21:25:57 DietPi kernel: [
] (snd_pcm_common_ioctl1+0x828/0xd4c) from [ ] (snd_pcm_playback_ioctl1+0x138/0x440) Feb 06 21:25:57 DietPi kernel: [ ] (snd_pcm_playback_ioctl1+0x138/0x440) from [<c0121e3 8>] (do_vfs_ioctl+0x80/0x598) Feb 06 21:25:57 DietPi kernel: [ ] (do_vfs_ioctl+0x80/0x598) from [ ] (SyS_ioc tl+0x6c/0x7c) Feb 06 21:25:57 DietPi kernel: [ ] (SyS_ioctl+0x6c/0x7c) from [ ] (__sys_trace _return+0x0/0x18) Feb 06 21:25:57 DietPi kernel: ---[ end trace 6268d48b9c1f131d ]--- Am 03.02.2021 um 01:09 schrieb MichaIng notifications@github.com:
The kernel is the same indeed, the official Allo Sparky SBC kernel https://github.com/sparkysbc/Linux including patches https://github.com/sparky-sbc/sparky-test. Could you give some details on the errors you face?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MichaIng/DietPi/issues/4080#issuecomment-772103321, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEC2TDX5EM7VOTFYGCDLAQTS5CH3JANCNFSM4W7SSPKQ.
MichaIng
commented
3 years ago Good that it runs stable so far. The kernel errors are not beautiful, but I'm not even sure if they are related to each other. Scheduling and IRQ handling seems to be involved, but I'm not great in reading this.
Could you print:
cat /proc/interrupts
cpuroot@DietPi:/mnt/dietpi_userdata# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
29: 6276247 6267591 6608169 8794807 GIC twd
30: 0 0 0 0 GIC owl_wdt
32: 1581015 0 1 0 GIC ethernet_mac
33: 0 0 0 0 GIC asoc_de
43: 9 0 0 0 GIC timer1_tick
55: 0 0 0 0 GIC xhci-hcd:usb1
56: 56 0 0 0 GIC aotg_hub_hcd:usb3
57: 4813928 0 0 0 GIC b0170000.i2c
58: 0 0 0 0 GIC b0174000.i2c
59: 2 0 0 0 GIC b0178000.i2c
64: 65 0 0 0 GIC
74: 1 0 0 0 GIC sdcard
76: 55961 0 0 0 GIC emmc
78: 0 0 0 0 GIC hdmidev
81: 0 0 0 0 GIC vce_isr
82: 0 0 0 0 GIC vde_isr
89: 20599 0 0 0 GIC owl_dma0
90: 0 0 0 0 GIC owl_dma1
91: 0 0 0 0 GIC owl_dma2
92: 0 0 0 0 GIC owl_dma3
93: 40053562 0 0 0 GIC aotg_hub_hcd:usb4
215: 0 0 0 0 owl_sirq_irq atc2603c
216: 0 0 0 0 atc2603c atc260x_onoff
217: 0 0 0 0 atc2603c RTC alarm
219: 0 0 0 0 atc2603c atc260x-irkeypad
IPI0: 0 0 0 0 CPU wakeup interrupts
IPI1: 0 0 0 0 Timer broadcast interrupts
IPI2: 196210 865331 498975 562729 Rescheduling interrupts
IPI3: 528 965 943 1058 Function call interrupts
IPI4: 480 8 11 5 Single function call interrupts
IPI5: 0 0 0 0 CPU stop interrupts
IPI6: 0 0 0 0 completion interrupts
IPI7: 0 0 0 0 CPU backtrace
Err: 0root@DietPi:/mnt/dietpi_userdata# cpu
─────────────────────────────────────────────────────
DietPi CPU Info
Use dietpi-config to change CPU / performance options
─────────────────────────────────────────────────────
Architecture | armv7l
Temperature | 45'C : 113'F (Optimal temperature)
Governor | conservative
Throttle up | 80% CPU usage
Current Freq Min Freq Max Freq
CPU0 | 504 MHz 240 MHz 1104 MHz
CPU1 | 504 MHz 240 MHz 1104 MHz
CPU2 | 504 MHz 240 MHz 1104 MHz
CPU3 | 504 MHz 240 MHz 1104 MHz
[ INFO ] DietPi-CPU_info | CPU current frequency, may be affected by this script, due to the processing required to run it.Okay, CPU temperature and governor work fine, also conservative should be, as the name implies, a conservative choice in regards to stability, as clock rates are adjusted in small steps.
As expected, most IRQs are handled by CPU0 only. Is it possible to change that on Sparky SBC?
echo 2 > /proc/irq/93/smp_affinityIf this succeeds (not every kernel allows it), USB port 4 interrupts should be handled by CPU1 then, so cat /proc/interrupts should show start counting in the CPU1 column:
93: 40053562 0 0 0 GIC aotg_hub_hcd:usb4This could enhance/speedup interrupt handling in general, when CPU0 is very busy already and needs to handle many interrupts then as well, but no guarantee it solves the kernel errors or make any other notable difference.
It can be applied at boot automatically:
echo 'f /proc/irq/93/smp_affinity - - - - 2' > /etc/tmpfiles.d/usb4_smp_affinity.confwhat we do now is far beyond my knowledge…..
root@DietPi:/mnt/dietpi_userdata# thin_repair /var/lib/docker/devicemapper/devicemapper/metadata no input file provided Usage: thin_repair [options] {device|file} Options: {-h|--help} {-i|--input} <input metadata (binary format)> {-o|--output} <output metadata (binary format)> {-V|--version}
root@DietPi:/mnt/dietpi_userdata# thin_check --clear-needs-check-flag /var/lib/docker/devicemapper/devicemapper/metadata examining superblock superblock is corrupt bad checksum in superblock, wanted 1490015127 root@DietPi:/mnt/dietpi_userdata#
root@DietPi:/mnt/dietpi_userdata# thin_check /var/lib/docker/devicemapper/devicemapper/metadata examining superblock superblock is corrupt bad checksum in superblock, wanted 1490015127
i copied the metadata file to metadata_old and reruns the thin_repair with the -i and -o option it worked but afterwards thin_check throws the same error like above
then i delete /var/lib/docker and restart docker - it doesn’t improve the situation a new install also fails…..
I stop now. perhaps it rely on sparky - i don’t know
i will setup the sparky with an older version of DietPi based on stretch. With that older version it is possible for me to prepare the system to be able to load the „old“ shared libs i need for idi-tidal-connect. buster is too new because the idi-streamer need libcurl3 instead of libcurl4
if have a raspy here with dietpi on buster 64 bit i try to install docker there. Maybe i have more luck
last question: the irq affinity ….. when i change this, could there be any risk for the stability of the system?
Am 07.02.2021 um 21:40 schrieb MichaIng notifications@github.com:
Okay, CPU temperature and governor work fine, also conservative should be, as the name implies, a conservative choice in regards to stability, as clock rates are adjusted in small steps.
As expected, most IRQs are handled by CPU0 only. Is it possible to change that on Sparky SBC?
echo 2 > /proc/irq/93/smp_affinity If this succeeds (not every kernel allows it), USB port 4 interrupts should be handled by CPU1 then, so cat /proc/interrupts should show start counting in the CPU1 column:
93: 40053562 0 0 0 GIC aotg_hub_hcd:usb4 This could enhance/speedup interrupt handling in general, when CPU0 is very busy already and needs to handle many interrupts then as well, but no guarantee it solves the kernel errors or make any other notable difference.
It can be applied at boot automatically:
echo 'f /proc/irq/93/smp_affinity - - - - 2' > /etc/tmpfiles.d/usb4_smp_affinity.conf — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MichaIng/DietPi/issues/4080#issuecomment-774756738, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEC2TDX2KRGCCZOECGWWVD3S533EHANCNFSM4W7SSPKQ.
MichaIng
commented
3 years ago This thin_repair stuff is also new to me. So it seems to require either -i and/or also -o to repair a metadata file. Indeed it's possible that the quite old kernel 3.10 starts to have issues with newer libraries/software, especially such like Docker which make use of kernel features. The good news that that everything that you can do with Docker, can be done without Docker as well. Only it requires some more install steps, in case 😉.
last question: the irq affinity ….. when i change this, could there be any risk for the stability of the system?
Only if you do it the opposite way round: Forcefully load all interrupts onto a single CPU. But we move one device from a more loaded CPU to a lesser loaded one (at least when it's about the interrupts), so either there is no difference, or a bit faster interrupt handling for this USB port. If the kernel cannot deal with it, writing to that file would simply fail. E.g. on RPi it's not possible to change it.
seniorgod
commented
3 years ago i give up here and move roll to dietpi 6.34.3 on stretch for sparky thanks for you help
MichaIng
commented
3 years ago Okay. Would be interesting btw if this solves the kernel errors. If they stay, we could consult Allo developers and with some luck a kernel patch could be supplied.
seniorgod
commented
3 years ago Dear all,
today my dietpi on sparky "crashed" during playback. Beside you find the kernel trace:
Apr 02 16:40:43 DietPi kernel: ------------[ cut here ]------------
Apr 02 16:40:43 DietPi kernel: WARNING: at /imxhdd/opt/1301TAG/sparky_volumio/kernel/net/sched/sch_generic.c:255 dev_watchdog+0x24c/0x26c()
Apr 02 16:40:43 DietPi kernel: NETDEV WATCHDOG: eth0 (owl-ethernet): transmit queue 0 timed out
Apr 02 16:40:43 DietPi kernel: Modules linked in: snd_usb_audio snd_hwdep snd_usbmidi_lib nls_cp437 ethernet spidev atc260x_irkeypad atc260x_cap_gauge autofs4
Apr 02 16:40:43 DietPi kernel: CPU: 0 PID: 0 Comm: swapper/0 Tainted: G W 3.10.38 #22
Apr 02 16:40:43 DietPi kernel: [<c0015b34>] (unwind_backtrace+0x0/0x138) from [<c001304c>] (show_stack+0x24/0x2c)
Apr 02 16:40:43 DietPi kernel: [<c001304c>] (show_stack+0x24/0x2c) from [<c002beb8>] (warn_slowpath_common+0x4c/0x6c)
Apr 02 16:40:43 DietPi kernel: [<c002beb8>] (warn_slowpath_common+0x4c/0x6c) from [<c002bf6c>] (warn_slowpath_fmt+0x30/0x40)
Apr 02 16:40:43 DietPi kernel: [<c002bf6c>] (warn_slowpath_fmt+0x30/0x40) from [<c05ea148>] (dev_watchdog+0x24c/0x26c)
Apr 02 16:40:43 DietPi kernel: [<c05ea148>] (dev_watchdog+0x24c/0x26c) from [<c003a2d8>] (call_timer_fn+0x3c/0x154)
Apr 02 16:40:43 DietPi kernel: [<c003a2d8>] (call_timer_fn+0x3c/0x154) from [<c003ac90>] (run_timer_softirq+0x1c0/0x2c4)
Apr 02 16:40:43 DietPi kernel: [<c003ac90>] (run_timer_softirq+0x1c0/0x2c4) from [<c00339e0>] (__do_softirq+0xf4/0x2a0)
Apr 02 16:40:43 DietPi kernel: [<c00339e0>] (__do_softirq+0xf4/0x2a0) from [<c0033c1c>] (do_softirq+0x4c/0x58)
Apr 02 16:40:43 DietPi kernel: [<c0033c1c>] (do_softirq+0x4c/0x58) from [<c0033e90>] (irq_exit+0x90/0xc8)
Apr 02 16:40:43 DietPi kernel: [<c0033e90>] (irq_exit+0x90/0xc8) from [<c000fbf8>] (handle_IRQ+0x3c/0x94)
Apr 02 16:40:43 DietPi kernel: [<c000fbf8>] (handle_IRQ+0x3c/0x94) from [<c00085e0>] (gic_handle_irq+0x28/0x5c)
Apr 02 16:40:43 DietPi kernel: [<c00085e0>] (gic_handle_irq+0x28/0x5c) from [<c000ef40>] (__irq_svc+0x40/0x70)
Apr 02 16:40:43 DietPi kernel: Exception stack(0xc0c77f68 to 0xc0c77fb0)
Apr 02 16:40:43 DietPi kernel: 7f60: ffffffed 00a37000 c0c8b6e4 00000000 c0c76000 c0c76000
Apr 02 16:40:43 DietPi kernel: 7f80: c0c76000 c0d29dec c0c89ed4 414fc091 c07c6920 c0d295bd 00000000 c0c77fb0
Apr 02 16:40:43 DietPi kernel: 7fa0: c0010050 c0010048 60000013 ffffffff
Apr 02 16:40:43 DietPi kernel: [<c000ef40>] (__irq_svc+0x40/0x70) from [<c0010048>] (arch_cpu_idle+0x28/0x38)
Apr 02 16:40:43 DietPi kernel: [<c0010048>] (arch_cpu_idle+0x28/0x38) from [<c00736b8>] (cpu_startup_entry+0x68/0x24c)
Apr 02 16:40:43 DietPi kernel: [<c00736b8>] (cpu_startup_entry+0x68/0x24c) from [<c0c00a34>] (start_kernel+0x2c4/0x320)
Apr 02 16:40:43 DietPi kernel: ---[ end trace ddc8d07941dc5ddd ]---See the kernel trace above - perhaps it helps. I will also open a ticket at the allo forum at audiophilestyle.com
MichaIng
commented
3 years ago It's the same trace you posted above already (the second one), only on CPU 0 instead of on CPU 2 🤔.
The Ethernet connection was lost/timed out. Similar report (CentOS, but doesn't matter as it's kernel-level): https://bugs.centos.org/view.php?id=6249
We updated the lastest Ethernet driver with DietPi v6.33: https://github.com/MichaIng/DietPi/blob/master/dietpi/patch_file#L2631-L2636 Here the related instructions/script that Allo suggests: https://github.com/sparky-sbc/sparky-test/tree/master/sparky-eth The Ethernet driver was something is the only thing that was updated multiple times recently: https://github.com/sparky-sbc/sparky-test/commits/master So that is what could be tested (using an older kernel module) or at least have a look into (Allo devs) when debugging the issue.
I'm linking your forum thread here as well, so we can follow easier: https://audiophilestyle.com/forums/topic/62542-music-playing-crashes-and-kernel-trace-on-sparkyusbridge/
Btw a good hint about the volumio kernel patch. To be true I never re-compiled the kernel of our Sparky SBC image, but only updated the system around it + DietPi updates ship individual kernel module updates, as suggested by Allo and shipped with the above sparky-test repository. Re-compiling the kernel + bootloader (+ initramfs) freshly could be a good step.
seniorgod
commented
3 years ago ok - in that case i close it and hope that Allo ist answering in their forum
MichaIng
commented
3 years ago Okay. Since I might loose track on closed issues, please report back when you have news about it, especially when there is something we can do, a kernel patch, rebuild or workaround.
bamyasi
commented
3 years ago Since there were no updates coming from Allo and since I've also lost ability to use my Sparky USBridge after DietPi update due to kernel crashes, could you please re-open the issue? It would be much easier to track it using GitHub since Allo does not have any decent tracking system for customers, only a forum.
bamyasi
commented
3 years ago I got a PM from Allo.com tech support on may 4, they were promising to look into it. But I would not hold my breath, it's a 2 years old model after all. Since I have already switched to using RPi4 currently, I would not mind closing this ticket again. If I hear from Allo.com soon I will let you know anyway.
MichaIng
commented
2 years ago @bamyasi Did you get any feedback from Allo support?
bamyasi
commented
2 years ago No feedback from Allo.com, I guess Sparky hardware is no longer supported. Personally, I have switched to a generic RPi4 based streamer running DietPi OS. Having zero problems so far.
MichaIng
commented
2 years ago At least the product page shows "discontinued", the Volumio and Max2Play images aren't offered anymore either. I was thinking about asking Allo for a sample Sparky SBC, if they still have one around, to try getting mainline kernel to run on it. It was reported here that it is supported by mainline kernel device tree: https://www.phoronix.com/scan.php?page=news_item&px=Linux-4.16-New-ARM-Hardware But actually I'm not sure whether this is correct, since the Actions Semiconductor S700 is an ARMv8 Cortex A53 SoC while Sparky SBC has an ARMv7 Cortex A9 SoC, not sure how that SoC is named. Probably not worth it to invest time, since the Sparky SBC has/had its value with the many available HATs/DACs, and with mainline Linux those (or most) won't work due to missing drivers/device tree overlays.
Dear all,
i use dietpi since years with my allo sparky und uxbridge (for better sound). Until last week i use dietpi based on stretch. I don't know if i use dietpi with an older Debian version other than stretch. I set up my system new with the base image of dietpi for stretch in September last year. From that point until last week it plays without a problem. Last weekend i set up the system new, based on the image for DietPi on buster. Now, two days later i face problems during playback Music. The system freezes with kernel traces.
Is there a new kernel in the image based on buster that changed something?