Switch to rpi-3.13.y branch for 3.13 kernel

[anaconda]

This is a note for mk01/whoever is providing the 3.13 kernel package in devel.

It appears xbian-package-kernel_1.3.7-3 is compiled from the rpi-3.13.y-next branch, which features the following:

• it's not getting updated, they instead update the rpi-3.13.y branch;
• it's missing some commits, see e.g. #550

Edit: popcornmix said it'd be better to switch to 3.14, as only "stable" (3.12) and "latest" (3.14) are officially supported.

[CurlyMoo]

Maybe you can try to compile that kernel yourself? https://github.com/xbianonpi/xbian/wiki/Compiling-the-Linux-kernel-&-Raspberry-Pi-firmware

Not sure how up-to-date this is...

[anaconda]

That guide works, that's how I'm compiling my kernel. I'm not sure though if the packaged kernel in devel has any additional patches, ... Also, they recently created the branch rpi-3.15.y, (I don't use devel) is devel always on the latest kernel? (= should we now use 3.15?)

[CurlyMoo]

XBian devel or RPi's devel?

The xbian-patches repository should contain all latest patches used. However, i also don't know what mk01 has done locally.

Maybe we can add that kernel to the devel xbian repository and we hear soon enough right?

[anaconda]

I'll test 3.15 for a few days. So far: * the HW random generator module (bcm2708-rng) leads to a hang modprobe; (fixed)

• OpenELEC is reporting unspecified "stability issues" with ALSA.

3.14 anyways shouldn't have these problems. (They just created the rpi-3.15.y branch yesterday, give them some time.)

[mk01]

I changed 3.13-next for 3.13.y few days ago as it was already strange to me there are no changes for few weeks. :)

I have a 3.14 kernel xbian package but I never published it (even not to devel). it would be more than welcome as 3.13 and 3.14 are much better for arm as the previous implementations. unfortunately it seems there are some changes on kernel -> userspace interfaces as in few minutes I found few base linux system tools which were not working as before (the binutils package was updated long ago but again Debian Wheezy is very much behind. In Jessie it was ok).

so until we don't want to provide our re-bundled newer versions, I do not recommend migration. if popcornmix is telling 3.12 and 3.14 will be the only supported, then let's wait until this happens (as raspberry will need to do this)

[mk01]

some of the problems are described here for instance http://archlinux.2023198.n4.nabble.com/Linux-3-14-in-testing-td4696891.html

[anaconda]

You'd probably want to test 3.15, I'm not seeing the issues reported in that Arch ML.

Also, this is his comment - as they created the branch rpi-3.15.y yesterday, that probably means 3.14 is no longer "officially" supported.

I'm not saying we should immediately (!) switch to 3.15 (for devel), but let's keep an eye on it, as it doesn't appear to have these issues.

Edit: this issue could be closed as you already switched to the right branch - if we don't want to leave it open to discuss 3.15.

[mk01]

I wan't to change for 3.13 (at least) as soon as possible. although my problems with corruption vanished with latest FW, 3.13 kernel and now removed emmc_pll_core=0, CurlyMo is still reporting even boot issues with this new combinations.

Although 3.12 is now outperformed with those new kernels, I don't remember any other kernel we kept so long as mainstream - it was VERY solid.

on the other hand with RPI new FW versions are very often bond to kernel part code changes - but currently I'm not aware of anything what should be pushing us (for instance any problems with AVRs and passthrough for some new models / users or similar).

maybe I will speak to popcornmix.

btw: you REALLY can't reproduce the documented problems? hard to believe. for instance "findmnt" not able to display it's standard informations is making quite much scripts and tools in XBian not working - btrfs-snap, some boot scripts are not doing their jobs correctly etc.

[anaconda]

I'm on 3.15, not 3.13.

But yes, I can reproduce the findmnt issue still on 3.15. For some strange reason I've read "mountpoint" in place of "findmnt", my bad.

grep'ed /etc for "findmnt": tested all mentions of "findmnt", this one is failing: /etc/apt/apt.conf.d/06xbian-btrfs:1:dpkg::Pre-Invoke {"[ ! -e /etc/default/xbian-snap ] || . /etc/default/xbian-snap; [ "$ENABLEDAPT" = yes ] || exit 0; z=$(findmnt -n | grep -m1 . | awk '{print $2}'); z=${z#*\[\/}; export z=${z%%\/*}; if [ -x /usr/sbin/btrfs-auto-snapshot ]; then btrfs-auto-snapshot snapshot -k $KEEPAPT -l apt-run $z ; fi ;" }; There's no alternative for this? /proc/self/mountinfo appears to be ok, maybe we could parse it directly? Edit: do we want "root" here? This appears to work:

xbian@xbian ~ $ z=$(findmnt -n --mtab | awk '{print $2}' | grep -m1 '\[/root/@\]') # This is the only part changed
xbian@xbian ~ $ z=${z#*[/}; export z=${z%%/*} # Unescaped
xbian@xbian ~ $ echo $z
root

Edit2: similar issue in /usr/sbin/btrfs-auto-snapshot line 198 in xbian-package-config-shell. Need to try --mtab here too.

As for the other issues, if you could give me some hints (where to look specifically), I'll test them all manually. Anyway, I went through my /var/log and I couldn't find anything failing.

[CurlyMoo]

What about letting @anaconda do the kernel compilation instead of you @mk01?

[mk01]

@anaconda

I was considering using of "alias" as hotfix and be able to deploy it (or meaningfully test) but generally I'm man of conformance and don't like deviance so much mostly because they tend to be source of nothing but troubles - specially if you don't keep documentation at least at the extend of minimum ISO norms requirements (what we don't)

@CurlyMoo sure, anything is welcomed change. but unfortunately process itself including packaging and repo deployment is time consuming process, boring and without added value to the one responsible or to user. and because we don't have resources to spend, I think now is the time to automate all such repetitive tasks and let's use our know & how to make product better. but still is valid let's move to anaconda responsibility for the are kernel & firmware.

@anaconda

about the alias - create /etc/profile.d/kernel-3.15-fix.sh and put into it

alias findmnt='findmnt --map'

then relogin and try just "findmnt"

[anaconda]

Assuming you meant --mtab:

xbian@xbian ~ $ sudo nano -c /etc/profile.d/kernel-3.15-fix.sh 
xbian@xbian ~ $ source /etc/profile.d/kernel-3.15-fix.sh 
xbian@xbian ~ $ findmnt
TARGET            SOURCE                FSTYPE   OPTIONS
/sys              sysfs                 sysfs    rw,nosuid,nodev,noexec,noatime
/proc             proc                  proc     rw,nosuid,nodev,noexec,noatime
/dev              udev                  devtmpfs rw,relatime,size=256k,nr_inodes=46020,mode=755
/dev/pts          devpts                devpts   rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000
/run              tmpfs                 tmpfs    rw,nosuid,noexec,noatime,size=19148k,mode=755
/                 /dev/sda1[/root/@]    btrfs    rw,noatime,thread_pool=2,compress=lzo,space_cache,autodefrag,commit=120
/boot             /dev/mmcblk0p1        vfat     ro,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,errors=remount-ro
/home             /dev/sda1[/home/@]    btrfs    rw,noatime,thread_pool=2,compress=lzo,space_cache,autodefrag,commit=120
/lib/modules      /dev/sda1[/modules/@] btrfs    rw,noatime,thread_pool=2,compress=lzo,space_cache,autodefrag,commit=120
/sys/kernel/debug                       debugfs  rw,relatime
/xbmc-backup      /dev/sda1[/data/@]    btrfs    rw,noatime,thread_pool=2,compress=lzo,space_cache,autodefrag,commit=120

You'd need to grep as the root subvolume is not the first entry, but as far as I know aliases don't work in scripts. I may be wrong on that.

[mk01]

cat << \EOF > script
#!/bin/bash
shopt -s expand_aliases
. /etc/profile.d/xbian-alias.sh

xl
EOF
chmod +x script

now do

./script

[mk01]

@anaconda

the actual order doesn't matter. findmnt is there to simplify things and not make it difficult and works like "df" so you can specify object directly:

findmnt /

[mk01]

what I obviously didn't know back then. the easy way not dependent on orders should look now

findmnt / -n -o source

[CurlyMoo]

I saw that the new 3.15 kernel got lz4 zram support. Anyone tested it yet?

[anaconda]

@mk01 I meant we can't just add an alias in /etc/profile.d and expect everything on the system use it. If only /etc/apt/apt.conf.d/06xbian-btrfs and /usr/sbin/btrfs-auto-snapshot have problems with it, they're under our control, so we can just change their code to use --mtab and the alias at this point is not needed.

Also, can you provide me the 3.13.11+ .config (I'm lazy and don't want to install it just to extract the config)? I can build a package for 3.15.0+ (using olddefconfig) so both of you can test it, if desired. / cc @CurlyMoo

My current kernel configuration has the vchiq module built-in and I don't see the problems you reported with it on 3.12 (it "resetting" the CPU frequency; though my kernel has powersave as the default/boot governor, otherwise my Raspberry Pi overclocked to 1 GHz won't boot, even on 3.12. I didn't report this as that's expected that some boards are more "sensible" than others). Let me know if you want it built-in so you can test it too.

[mk01]

I pushed 3.15.1+ into staging. for findmnt I created simple wrapper for now. Yes, vchiq compiled in won't reset the freqs because it is loaded with kernel ;). I don't know the internals (and are not the point here) but probably vchiq is loaded (when build-in) before kernel is playing with freq and governors - that's why this effect is not seen in that case. I'm aware of that.

what you can test is vchiq (in-kernel) with kexec load & reboot into another (or even the same) kernel. with 3.12 (and before), vchiq is not releasing resources thus kexec boot will succeed basically, but none of services working via vchiq are accessible (basically even "vcgencmd version" will fail).

[anaconda]

Will test later today or tomorrow. I'm not going to install XBian kernels for the reason I explained in the previous comment (my Pi not booting with a 1 GHz overclock if the boot/default governor isn't "powersave"), but I'd like to diff the .config to find differences with mine - may I have it uploaded somewhere? :-)

[mk01]

xbian-patches -> kernel -> bcm ...

and just oldconfig

[mk01]

specially for you I pushed config 3.15 (which is part of .deb package). :)

[anaconda]

Aha, thank you!

At a first look (didn't diff yet) I see we had CONFIG_CC_STACKPROTECTOR=y in 3.12, but it's not set in 3.15. The setting was renamed to CONFIG_CC_STACKPROTECTOR_REGULAR in 3.15 and the kernel configuration scripts don't know about renamed settings, it needs to be re-enabled manually.

[CurlyMoo]

Please do. Stackprotection is pretty useful in C development.

[mk01]

changed. tomorrow package + headers.

[anaconda]

@mk01 on a side note, I replied to your e-mail from this morning. And I also checked the MBP, I don't have anything there too. Now that I remember, I deleted the key as last time I tried to login it was refused by the server (and I thought that VM was just wiped).

Sorry for hijacking this GitHub issue, but could you please participate in these forum threads: http://forum.xbian.org/thread-2353.html http://forum.xbian.org/thread-2334.html

[anaconda]

3.16 was recently released and - see first post - "latest" is now 3.16. It may be worth replacing 3.15 with 3.16, as it also fixes some nasty Btrfs corruptions (!).

We'd also need to unset MEMCG_KMEM: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=2ee06468702e0742114823a537510cd6f038cacc

[mk01]

@anaconda

do we speak still about the same twice!!! regressed situation with broken "atomic" snapshots containing unfinished transactions - leaving non resolvable trees? that was problem with 3.12 (our tree is fixed, then with 3.14 again), our 3.15 is ok. with last regress I was checking the code for all our kernels.

[anaconda]

No no, anything specific - that should be read as "it had lots of commits and that's our default FS": https://lkml.org/lkml/2014/6/11/360

(And I just noticed that 3.15 was EOL'd: https://kernel.org; https://lkml.org/lkml/2014/8/14/7)

[CurlyMoo]

We're already on 3.16 in devel so fixed :)