search

Hexxeh / rpi-update

An easier way to update the firmware of your Raspberry Pi
MIT License
1.94k stars 232 forks source link

Repeated kernel issues after 2.7 days runtime in 4.9 line (here 4.9.14+) #238

Closed ghost closed 7 years ago

ghost commented 7 years ago

After upgrading to the 4.9 kernel line, my weather station keeps locking up after 2 or 3 days. Sometimes I cannot log in after this, but now I managed to scrape a few details.

No space/ memory/ swap issues. Used to work in same setup for over two years (with occasional Raspbian updates and rpi-update kernel updates). Moved to btrfs filesystem at around kernel 4.5 and it was fine until 4.9 came along. (Note, swap is on a separate partition.)

Any ideas how to approach this? Would be happy to try out a few things. It seems to be ok after a reboot. Will update to latest kernel 4.9.16 now.

Best regards, Max

pi@raspberrypi ~ $ uname -a Linux raspberrypi 4.9.14+ #977 Mon Mar 13 18:21:04 GMT 2017 armv6l GNU/Linux

dmesg except: (Tofu below are square brackets)

[233136.361008] octave-cli: page allocation stalls for 10660ms, order:2, mode:0x24000c0(GFP_KERNEL) [233136.370269] CPU: 0 PID: 21214 Comm: octave-cli Not tainted 4.9.14+ #977 [233136.377216] Hardware name: BCM2835 [233136.380868] [] (unwind_backtrace) from [] (show_stack+0x20/0x24) [233136.388960] [] (show_stack) from [] (dump_stack+0x20/0x28) [233136.396536] [] (dump_stack) from [] (warn_alloc+0xf8/0x120) [233136.404164] [] (warn_alloc) from [] (alloc_pages_nodemask+0xaec/0xd04) [233136.415671] [] (alloc_pages_nodemask) from [] (get_free_pages+0x20/0x70) [233136.430524] [] (get_free_pages) from [] (pgd_alloc+0x24/0x104) [233136.444421] [] (pgd_alloc) from [] (mm_init+0xcc/0x168) [233136.454699] [] (mm_init) from [] (copy_process.part.5+0x968/0x16c8) [233136.468976] [] (copy_process.part.5) from [] (_do_fork+0xb4/0x3a0) [233136.483236] [] (_do_fork) from [] (SyS_clone+0x30/0x38) [233136.493652] [] (SyS_clone) from [] (ret_fast_syscall+0x0/0x1c) [233136.507669] Mem-Info: [233136.513262] active_anon:7030 inactive_anon:7049 isolated_anon:1 active_file:19465 inactive_file:17091 isolated_file:31 unevictable:0 dirty:54 writeback:3 unstable:0 slab_reclaimable:2189 slab_unreclaimable:1903 mapped:10587 shmem:2810 pagetables:360 bounce:0 free:590 free_pcp:0 free_cma:0 [233136.564313] Node 0 active_anon:28120kB inactive_anon:28196kB active_file:77860kB inactive_file:68364kB unevictable:0kB isolated(anon):4kB isolated(file):124kB mapped:42348kB dirty:216kB writeback:12kB shmem:11240kB writeback_tmp:0kB unstable:0kB pages_scanned:0 all_unreclaimable? no [233136.600418] Normal free:2360kB min:1868kB low:2332kB high:2796kB active_anon:28120kB inactive_anon:28196kB active_file:77860kB inactive_file:68364kB unevictable:0kB writepending:228kB present:245760kB managed:233620kB mlocked:0kB slab_reclaimable:8756kB slab_unreclaimable:7612kB kernel_stack:1760kB pagetables:1440kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB [233136.645890] lowmem_reserve[]: 0 0 [233136.651882] Normal: 5544kB (UM) 188kB (UM) 016kB 032kB 064kB 0128kB 0256kB 0512kB 01024kB 02048kB 0*4096kB = 2360kB [233136.668449] 39747 total pagecache pages [233136.673433] 368 pages in swap cache [233136.679284] Swap cache stats: add 9416, delete 9048, find 187356/188426 [233136.688374] Free swap = 250180kB [233136.694109] Total swap = 265212kB [233136.699709] 61440 pages RAM [233136.704778] 0 pages HighMem/MovableOnly [233136.710806] 3035 pages reserved [233136.716150] 2048 pages cma reserved

culprit is 21214 octave-cli

pi@raspberrypi ~ $ ps aux --forest|cat USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 2 0.0 0.0 0 0 ? S Mar16 0:00 [kthreadd] root 3 0.0 0.0 0 0 ? S Mar16 1:26 _ [ksoftirqd/0] root 5 0.0 0.0 0 0 ? S< Mar16 0:00 _ [kworker/0:0H] root 7 0.0 0.0 0 0 ? S< Mar16 0:00 _ [lru-add-drain] root 8 0.0 0.0 0 0 ? S Mar16 0:00 _ [kdevtmpfs] root 9 0.0 0.0 0 0 ? S< Mar16 0:00 _ [netns] root 10 0.0 0.0 0 0 ? S Mar16 0:00 _ [khungtaskd] root 11 0.0 0.0 0 0 ? S Mar16 0:00 _ [oomreaper] root 12 0.0 0.0 0 0 ? S< Mar16 0:00 _ [writeback] root 13 0.0 0.0 0 0 ? S Mar16 0:03 _ [kcompactd0] root 14 0.0 0.0 0 0 ? S< Mar16 0:00 _ [crypto] root 15 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 16 0.0 0.0 0 0 ? S< Mar16 0:00 _ [kblockd] root 17 0.0 0.0 0 0 ? S< Mar16 0:00 _ [watchdogd] root 19 0.0 0.0 0 0 ? S< Mar16 0:00 _ [rpciod] root 20 0.0 0.0 0 0 ? S< Mar16 0:00 _ [xprtiod] root 23 0.0 0.0 0 0 ? S Mar16 1:13 _ [kswapd0] root 24 0.0 0.0 0 0 ? S< Mar16 0:00 _ [nfsiod] root 34 0.0 0.0 0 0 ? S< Mar16 0:00 _ [kthrotld] root 35 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 36 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 37 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 38 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 39 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 40 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 41 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 42 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 43 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 44 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 45 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 46 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 47 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 48 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 49 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 50 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 51 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 52 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 53 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 54 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 55 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 56 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 57 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 58 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 59 0.0 0.0 0 0 ? S< Mar16 0:00 \ [iscsieh] root 60 0.0 0.0 0 0 ? S< Mar16 0:00 \ [dwcotg] root 61 0.0 0.0 0 0 ? S< Mar16 0:00 _ [DWC Notificatio] root 62 0.0 0.0 0 0 ? S< Mar16 0:00 _ [VCHIQ-0] root 63 0.0 0.0 0 0 ? S< Mar16 0:00 _ [VCHIQr-0] root 64 0.0 0.0 0 0 ? S< Mar16 0:00 _ [VCHIQs-0] root 65 0.0 0.0 0 0 ? S Mar16 0:00 _ [VCHIQka-0] root 77 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 80 0.0 0.0 0 0 ? S Mar16 3:15 _ [mmcqd/0] root 112 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 141 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-worker] root 142 0.0 0.0 0 0 ? S< Mar16 0:00 _ [kworker/u3:0] root 143 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-worker-hi] root 144 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-delalloc] root 145 0.0 0.0 0 0 ? S< Mar16 0:00 \ [btrfs-flushdel] root 146 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-cache] root 147 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-submit] root 148 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-fixup] root 149 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-endio] root 150 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-endio-met] root 151 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-endio-met] root 152 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-endio-rai] root 153 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-endio-rep] root 154 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-rmw] root 155 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-endio-wri] root 156 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-freespace] root 157 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-delayed-m] root 158 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-readahead] root 159 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-qgroup-re] root 160 0.0 0.0 0 0 ? S< Mar16 0:00 _ [btrfs-extent-re] root 161 0.0 0.0 0 0 ? S Mar16 0:00 _ [btrfs-cleaner] root 162 0.1 0.0 0 0 ? S Mar16 4:46 _ [btrfs-transacti] root 178 0.0 0.0 0 0 ? S< Mar16 0:02 _ [kworker/0:1H] root 181 0.0 0.0 0 0 ? S< Mar16 0:00 \ [ipv6addrconf] root 211 0.9 0.0 0 0 ? S Mar16 40:25 \ [w1_busmaster1] root 294 0.0 0.0 0 0 ? S Mar16 0:00 _ [spi0] root 380 0.0 0.0 0 0 ? S< Mar16 0:00 _ [cfg80211] root 386 0.0 0.0 0 0 ? S< Mar16 0:58 _ [kworker/u3:1] root 467 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 468 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 469 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 470 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 471 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 472 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 473 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 474 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 475 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 476 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 477 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 478 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 479 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 480 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 481 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 482 0.0 0.0 0 0 ? S< Mar16 0:00 _ [bioset] root 19619 0.0 0.0 0 0 ? S 11:55 0:03 _ [kworker/u2:5] root 21696 0.0 0.0 0 0 ? D 12:30 0:04 | _ [kworker/u2:5] root 22160 0.0 0.0 0 0 ? S 18:12 0:00 _ [kworker/u2:0] root 22178 0.0 0.0 0 0 ? S 18:30 0:01 _ [kworker/u2:2] root 22193 0.1 0.0 0 0 ? S 18:43 0:02 _ [kworker/0:3] root 22196 0.1 0.0 0 0 ? S 18:48 0:01 _ [kworker/0:2] root 22205 0.0 0.0 0 0 ? S 18:58 0:00 _ [kworker/u2:4] root 22271 0.1 0.0 0 0 ? S 19:07 0:00 _ [kworker/0:0] root 22274 0.0 0.0 0 0 ? S 19:10 0:00 _ [kworker/u2:1] root 22280 0.0 0.0 0 0 ? S 19:14 0:00 \ [kworker/0:1] root 1 0.0 1.4 5464 3396 ? Ss Mar16 0:26 /sbin/init root 224 0.0 0.8 8104 2092 ? Ss Mar16 2:07 /lib/systemd/systemd-journald root 230 0.0 0.6 11988 1468 ? Ss Mar16 0:00 /lib/systemd/systemd-udevd root 511 0.0 1.2 7144 2816 ? Ss Mar16 0:23 /sbin/wpa_supplicant -s -B -P /run/wpa_supplicant.wlan0.pid -i wlan0 -W -D nl80211,wext -c /etc/wpa_supplicant/wpasupplicant.conf root 568 0.0 1.0 31240 2352 ? Ssl Mar16 0:25 /usr/sbin/rsyslogd -n root 571 0.0 0.9 7856 2300 ? Ss Mar16 0:08 /usr/sbin/sshd -D root 21690 0.0 1.8 9200 4424 ? Ds 12:29 0:05 _ sshd: [accepted]
root 22201 0.0 2.3 12600 5472 ? Ss 18:58 0:00 _ sshd: pi [priv]
pi 22215 0.1 1.5 12740 3632 ? S 18:58 0:01 _ sshd: pi@pts/0
pi 22217 0.2 1.9 6340 4508 pts/0 Ss 18:58 0:02 _ -bash pi 22284 0.0 0.9 4884 2316 pts/0 R+ 19:16 0:00 _ ps aux --forest pi 22285 0.0 0.6 3540 1452 pts/0 S+ 19:16 0:00 _ cat root 576 0.0 1.3 40200 3264 ? Ssl Mar16 0:00 /usr/sbin/ModemManager avahi 577 0.0 0.9 3864 2132 ? Ss Mar16 0:04 avahi-daemon: running [raspberrypi-2.local] avahi 626 0.0 0.3 3864 824 ? S Mar16 0:00 \ avahi-daemon: chroot helper root 585 0.0 0.8 3844 2092 ? Ss Mar16 0:03 /lib/systemd/systemd-logind message+ 606 0.0 0.9 5736 2164 ? Ss Mar16 0:01 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation nobody 620 0.0 0.4 2280 1140 ? Ss Mar16 0:05 /usr/sbin/thd --daemon --triggers /etc/triggerhappy/triggers.d/ --socket /var/run/thd.socket --pidfile /var/run/thd.pid --user nobody /dev/input/event* ntp 633 0.0 0.8 5676 2064 ? Ss Mar16 0:53 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 102:104 root 639 0.0 0.6 12368 1496 ? Ss Mar16 0:00 /usr/sbin/cupsd -f root 641 0.0 0.9 10204 2316 ? Ss Mar16 0:00 /usr/sbin/cups-browsed root 653 0.0 1.5 39644 3604 ? Ssl Mar16 0:00 /usr/lib/policykit-1/polkitd --no-debug root 666 0.0 0.5 4040 1232 tty1 Ss+ Mar16 0:00 /sbin/agetty --noclear tty1 linux root 667 0.0 0.5 3860 1236 ? Ss+ Mar16 0:00 /sbin/agetty --keep-baud 115200 38400 9600 ttyAMA0 vt102 www-data 693 0.0 0.9 8140 2176 ? Ss Mar16 0:23 /usr/sbin/lighttpd -D -f /etc/lighttpd/lighttpd.conf root 713 0.0 0.5 2576 1228 ? Ss Mar16 0:11 /sbin/wpa_cli -B -P /run/wpa_action.wlan0.pid -i wlan0 -p /var/run/wpa_supplicant -a /sbin/wpaaction root 751 0.0 0.7 8716 1664 ? Ss Mar16 0:00 dhclient -v -pf /run/dhclient.wlan0.pid -lf /var/lib/dhcp/dhclient.wlan0.leases wlan0 root 865 0.0 1.7 30616 3984 ? Ssl Mar16 0:00 /usr/sbin/console-kit-daemon --no-daemon root 2656 0.0 0.7 5088 1648 ? Ds Mar16 0:18 /usr/sbin/cron -f root 21191 0.0 1.1 7016 2672 ? S 12:19 0:00 _ /usr/sbin/CRON -f root 21204 0.0 0.4 1900 1104 ? Ss 12:19 0:00 | _ /bin/sh -c cd /root && ./sensorplots root 21206 0.0 1.1 4604 2636 ? S 12:19 0:00 | _ /bin/bash ./sensorplots root 21214 0.0 15.6 126792 36612 ? D 12:19 0:13 | _ /usr/bin/octave-cli -H /root/sensorplots-20170319.m root 21614 0.0 0.0 0 0 ? Z 12:29 0:00 _ [cron] root 21616 0.0 1.1 7016 2672 ? S 12:29 0:00 _ /usr/sbin/CRON -f root 21625 0.0 0.4 1900 1116 ? Ss 12:29 0:00 | _ /bin/sh -c cd /root && ./sensorcron root 21629 0.0 1.1 4604 2648 ? S 12:29 0:00 | _ /bin/bash ./sensorcron root 21676 0.0 0.7 4604 1820 ? D 12:29 0:04 | _ /bin/bash ./sensorcron root 21695 0.0 1.0 7016 2528 ? D 12:30 0:00 _ /usr/sbin/CRON -f pi 22206 0.0 1.4 4964 3320 ? Ss 18:58 0:00 /lib/systemd/systemd --user pi 22209 0.0 0.8 7352 2056 ? S 18:58 0:00 \ (sd-pam)

popcornmix commented 7 years ago

This repo is just for the rpi-update utility. Could you recreate the issue here: https://github.com/raspberrypi/linux/issues where it's more likely to be seen by the right people.