Update to Linux Kernel 4.1.5+ breaks all network connections

[Smultie]

Subject says it all...

[mkreisl]

Hmm, I assume you are updating kernel to 4.1.5+-1439833181 on a RPi2/Jessie?

I have exactly this version running on my RPi2, both devices eth0 and wlan0 are working properly

[Smultie]

Yeah, that's the version. Bunch of updates today, fucked it all up.

[Smultie]

Could you please come to IRC to see how I can fix this... ?

[Smultie]

Welp, the 6th reboot somehow made it all work again..... really strange

[Smultie]

It currently looks like it depends on the gods whether or not the device starts with or without network. It looks like a sudo reboot will almost always result in no network, while pulling the plug often results in an available network....

Please let me know if I can somehow help with bugtracking this strange one...

[mkreisl]

Logfiles would be helpful, but it's difficult to do this in case of the error if no keyboard exists. Probably the eth driver (smsc95xx) does sometimes (or mostly) not initalize the hardware correctly

[Smultie]

What specific logfiles would be helpful?

[mkreisl]

Usually dmesg could show the issue

[Smultie]

http://pastebin.com/2nVgTP6U

How can I post an older one?

[mkreisl]

No way to post older ones, because this is the kernel ring buffer, means dmesg dumps all messages which resides in this buffer

This log looks ok, smsc95xx driver will be loaded and link is up, can't see any error

That's what I meant with but it's difficult to do this in case of the error if no keyboard exists

[Smultie]

Yeah, that's why I asked if it's possible to see older dmesg files. Cause once it works, I can access dmesg, if the network doesn't work, I can't access dmesg.

A bit of a catch-22.

[mkreisl]

what you can do is adding following line to /etc/init/xbmc.conf before xbmc is started (insert it before schedtool ...)

dmesg > /tmp/dmesg-$(date +%s).log

[Smultie]

Would this be okay?

#!upstart
description "xbmc"

env USER=xbian
env HOME=/home/xbian
env GROUP=xbian
env DAEMON="/usr/local/lib/xbmc/xbmc.bin"
env DAEMON_ARGS="--standalone -fs"
env DAEMON_LIRC="/run/lirc/lircd"
env PPLAYER='-3'
env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
env SCREENOFF=no
env ZTEMPXBMC=0
env DEBUG=''
env xs=Quit

dmesg > /tmp/dmesg-$(date +%s).log

start on started xbmc-preload and started dbus and (started autofs or xbian-nowait)
stop on starting rc RUNLEVEL=[06]

[mkreisl]

No sure, better here

    if [ -d /usr/lib/xbmc ]; then
        export LD_LIBRARY_PATH=/usr/lib/xbmc:$(sed -e '/^$\|^#/d' /etc/ld.so.conf.d/*.conf | sed ':a;N;$!ba;s/\n/:/g')
    fi

    dmesg > /tmp/dmesg-$(date +%s).log # dump kernel logs before starting kodi

    if [ "$DPRIORITY" = yes ]; then
        exec schedtool -R -p 49 -e sudo -u $USER -g $GROUP -s $DAEMON $DAEMON_ARGS  $DAEMON_LIRC $DEBUG 2>&1 | grep -v ES30 || :
    else
        sudo -u $USER -g $GROUP -s $DAEMON $DAEMON_ARGS  $DAEMON_LIRC $DEBUG 2>&1
    fi

[Smultie]

Thanks, done as you suggested. Will see if we find anything interesting. If my verrrryyyy limited knowledge of Linux helps me a bit, this will post a dmesg file with some date-time stuff added to the /tmp folder, right?

Will that file still be there after a reboot?

[mkreisl]

If my verrrryyyy limited knowledge of Linux helps me a bit, this will post a dmesg file with some date-time stuff

right, the filename contains the seconds since 1.1.1970

Will that file still be there after a reboot?

Oh, sorry, Xbian flushes /tmp during each reboot. So I tested /var/tmp, this folder won't be flushed

[bellini666]

I also had this problem when I tried to upgrade my rpi2. It was a real pain to get it back to work since I could not connect via ssh and ctrl+alt+f<n> would not drop to a console (anyone knows why it doesn't work?)

What I did to go back was to extract the kernel files from linux-image-bcm2836=4.1.4+-1439142124, put them in /boot, did what the postinst would do (rename kernel.img and System.map) and with that I could go back to a working state.

I can probably help finding anything relevant on the logs when I have some time on the next week if the issue is not solved until then. But it would be really nice if I could ctrl+alt+f<n> with an usb keyboard to get those logs.

[Smultie]

So, you agree there's something not completely right with kernel 4.1.5?

On 20 August 2015 at 14:32, Thiago Bellini notifications@github.com wrote:

I also had this problem when I tried to upgrade my rpi2. It was a real pain to get it back to work since I could not connect via ssh and ctrl+alt+f would not drop to a console (anyone knows why it doesn't work?)

What I did to go back was to extract the kernel files from linux-image-bcm2836=4.1.4+-1439142124, put them in /boot, did what the postinst would do (rename kernel.img and System.map) and with that I could go back to a working state.

I can probably help finding anything relevant on the logs when I have some time on the next week if the issue is not solved until then. But it would be really nice if I could ctrl+alt+f with an usb keyboard to get those logs.

— Reply to this email directly or view it on GitHub https://github.com/xbianonpi/xbian/issues/755#issuecomment-132992231.

[mkreisl]

So what do do think, it is better to revert to 4.1.4?

I can't find any issue, because this kernel works well on my RPi1 and RPi2

[bellini666]

@mkreisl that is strange. 2 issues (together with https://github.com/xbianonpi/xbian/issues/756) that me and, in this case, another person is experiencing and you are not.

I'm wondering what is the variable that is differing between our installations that is making stuff break for me and work for you. Just to comment about my installation:

I did the actual one ~4 months ago. I got the latest stable image in that time. After we started having the discussion about migrating to jessie, I changed my sources (as you can see on https://github.com/xbianonpi/xbian/issues/756#issuecomment-132992882) and upgraded to jessie.

Since I didn't have any issues with my rpi2 so far, I haven't had to do any manual modifications on any system configuration.

So what do do think, it is better to revert to 4.1.4?

Maybe, but it would be even better to understand that discrepancy between our systems. Or else we are fated to break the system for some users from time to time without even knowing it (since atm we assume that all installations should behave the same)

[bellini666]

@mkreisl based on your comment here, one difference between our tests was that your installation was fresh and mine was an upgraded one from some time ago.

So, one thing that might be making a difference is something that the upgrade process failed to do but the image is doing? Initially that doesn't make too much sense, but there are some cases that it could be an issue. For example, when we only replace a config file if it doesn't exist and things like that. It would make a file be totally different from an upgraded system and a fresh install. Not that it would be the issue here, but it is an example of how things could be different on both installations.

Can you provide me the link to download that RPI2 image? Also, if you want, I can create an image of my installation so you can test with a replica of it and see if you can reproduce the problem with it.

[mkreisl]

The images with jessie and kodi 15.1 are already on sourceforge, you can download them from there. They were built last night

[mkreisl]

A lot of people had already downloaded them. Hopefully they don't have such network and cec problems ...

[Smultie]

Looking at the file sizes of the 4.xx kernels, I noticed that 4.1.5 is significantly smaller than the rest of 'em. Could be nothing, could be something. Maybe somebody with a bit of knowledge kan take a look at it?

http://apt1.xbian.org/pool/stable/rpi2-jessie/l/linux-image-bcm2836/

[Smultie]

@hackedbellini What would be the steps needed to get back to the 4.1.4 kernel? Also, do I need to downgrade any other packages related?

[Smultie]

Alright, some new updates I see. Let's see how they work out...

[mkreisl]

kernel has been build now on our build server. Hopefully he does a better job :smile:

[Smultie]

Well, first reboot works fine, so, thus far it's been working okay ;)

On 20 August 2015 at 20:51, Manfred Kreisl notifications@github.com wrote:

kernel has been build on our build server. Hopefully he does a better job [image: :smile:]

— Reply to this email directly or view it on GitHub https://github.com/xbianonpi/xbian/issues/755#issuecomment-133117020.

[mkreisl]

sounds good

[bellini666]

@mkreisl FYI the new build also fixes the issue for me

[Smultie]

Just wondering; what was the bug?

[mkreisl]

No idea. Very strange. Because the first version of 4.1.5, build on my local system, worked w/o any issues on my RPi's