System freeze on btrfs scrub

[wlatendresse]

[b]Software[/b] XBian version: 1.0 (knockout) (kernel: Linux 3.18.8+) XBMC/Kodi version: Kodi 14.2-RC1 Overclock settings: default

[b]Hardware[/b] Raspberry Pi 2 Model B 1 GB: Power supply rating: 2.5A 5V Logilink PA0062 SD card size and make/type: 32GB microSDHC, Sony UHS-I Network (Ethernet or wireless): Ethernet 100M Connected devices (TV, USB, network storage, ...): Samsung UE55C6200 and Denon AVR-4310

[b]Log files[/b] Link to logfile(s): There are NO logs since the system simply freezes and it is not possible to get ANY logs anymore.

[b]Problem description:[/b] System freeze nearly allways when simply issuing the command "btrfs scrub start /" as root.

[b]How to reproduce:[/b] Issue the command mentioned above as root.

[wlatendresse]

And yeah, this should NOT be connected to the power source in ANY WAY, since otherwise the Raspi2 would display it's coloured square in the right upper corner ;-)

[CurlyMoo]

Why do you want to do scrubs in the first place?

[wlatendresse]

Why is THAT important? It still should not crash the system, I guess?

[CurlyMoo]

Ofc, but we need to know the specific use case in which this happens. So there probably is a reason you wanted to start a scrub.

[wlatendresse]

OK, the reason is very simple: I am using transmission with some very bis files (about 22GB) but when I destry the automatically created btrfs images, because they consume very much memory on my SD, the free space is NOT automatically regained. A scrub helps here, but ONLY if it does not crash, which it unfortunately does in 90% of all runs.

[CurlyMoo]

That's a lot of important info. Where you do store the downloaded files? All crashes we see reported are related to downloading.

[wlatendresse]

Directly on the SD card, did not change any of transmission paths. So it's /home/xbian/downloads , .../incomlete and .../torrents. ;-)

[wlatendresse]

Well, but this one is related to SCRUBbing ;-)

[wlatendresse]

BTW: Thanks for the great work nevertheless, IMHO much better than OpenELEC ;-)

[CurlyMoo]

Well, but this one is related to SCRUBbing ;-)

That's what you as end user might think, but it's not always that simple.

As a start i would advice you to use an external storage and disable auto snapshots creation. The SD card on a Raspberry Pi is just not made for such a load.

[wlatendresse]

I once work for a technical leader in a software company (yes I am a software developer myself) wo had very simple and clear rules about software development:

1. Find the BUGS
2. Find the Bugs
3. Continue with the rest ...

The software they produce is still pretty successfull for THIS SIMPLE REASON! IMHO you are doing something basic very wrong, when you simply ignore bugs ;-) How cna I help? Is there any kernel which will produce more information for xbian?

[CurlyMoo]

I understand but the core issue is we're running XBMC / Kodi from an SD card on a device that was never meant for it. Various OS' have found away around it. OE runs mainly from RAM, XBian uses BTRFS.

We want to find the bug, i'm just saying how you can get a working environment in the meanwhile.

How cna I help? Is there any kernel which will produce more information for xbian?

Join the kernel development team. We always need more professionals and the latest RPi kernels are bugging us for too long already.

[wlatendresse]

I really do NOT think this is a speed issue, since the appriprate system calls would simply block in that case. And there are NO issues on my other devices which run linux as well (BananaPi on Debian with OwnCloud, RasPi 1A on raspbian as router / firewall and plenty other things), all of them using btrfs as well and NO EXTERNAL DEVICES for files etc ;-)

[wlatendresse]

BTW: I barely have enough time to implement all my own ideas, unfortunately I am not sure that joining the kernel hackers would be much productive for all of us right now ;-) But I will try to get some info HOW this issue can be debugged because it REALLY bites me in my ass ;-)

[wlatendresse]

Any suggestions and pointers for remore kernel debugging (packages, HOWTOs ...) ?

[f1vefour]

The thing is we are using the lz4 btrfs patch and I am sure this is why these issues are appearing, there is some sort of incompatibility that isn't apparent. I tried communicating with the author but never got a response.

[CurlyMoo]

If you want to test, just start a clean image with lzop instead of lz4, that will give you another compression natively.

[f1vefour]

I don't know how to do that, I can build an unpatched kernel but I don't know how to build an image without lz4 being enabled. I'm still out of town and have no steady Internet access to do anything that takes any amount of time. Someone else will have to do it or it will have to wait until I return.

[CurlyMoo]

telnet zswap.enabled=1 zswap.compressor=lz4 sdhci-bcm2708.sync_after_dma=0 dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootflags=subvol=root/@,thread_pool=2,autodefrag,compress=lz4,commit=120 rootfstype=btrfs rootwait smsc95xx.turbo_mode=N logo.nologo quiet noswap loglevel=0 mod_scsi.scan=sync partswap startevent=mountall splash nohdparm --startup-event mountall

telnet zswap.enabled=1 zswap.compressor=lz4 sdhci-bcm2708.sync_after_dma=0 dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootflags=subvol=root/@,thread_pool=2,autodefrag,compress=lzo,commit=120 rootfstype=btrfs rootwait smsc95xx.turbo_mode=N logo.nologo quiet noswap loglevel=0 mod_scsi.scan=sync partswap startevent=mountall splash nohdparm --startup-event mountall

[f1vefour]

So just changing the boot will suffice? Can you do this on an already installed system?

[CurlyMoo]

Yes, but i think it will then only use lzo for new data.

[CurlyMoo]

Please try compress=lzo for btrfs and reboot (/boot/cmdline.txt)

[f1vefour]

I need someone with an RPi 2 to test this kernel, I'm hoping that the 3.18.y branch is simply buggy and 3.19.y is better.

https://db.tt/4xmGNDk2

Install by doing:

sudo cp /boot/kernel.img /boot/kernel.good
sudo dpkg -i linux-image-bcm2836_3.19.3+_armhf.deb

[f1vefour]

I'm working on a Pi 1 version also, will link when finished.

[wlatendresse]

Will have a look ASAP, probably in some hours ;-)

[wlatendresse]

After installation of the package and reboot the system will not even boot correctly anymore :-( . However I changed cmdline.txt to use lzo compression as well. Will have a closer look now ...

[wlatendresse]

Did you forget to include the multimediacard-driver? I don't see any /dev/mmc* devices and this seems to be the reason, the system will not boot.

[wlatendresse]

Mouse not working as well ("Device /dev/input/mouse0 not suitable"). Root partition /dev/mmcblk0p2 missing mount: special device /dev/mmcblk0p1 does not exist mount: special device /dev/mmcblk0p2 does not exist

Yout image is not working for me f1vefour :-(

[f1vefour]

Like I said it is untested since I'm away from my Pi's.

Which kernel did you install and on Pi 1 or 2?

[wlatendresse]

Yeah, you said, and I replied, was no complain, even if it may have sounded that way ;-)

linux-image-bcm28363.19.3+…rmhf.deb on Pi2.

[f1vefour]

I don't know why you have all those issues as it's using the same basic configuration the previous kernel is using.

When I can personally test I will make a new kernel, you can rename /boot/kernel.good to /boot/kernel.img on your PC and you will be using the previous kernel.

[wlatendresse]

If I had the time I would dive into the problem, but unfortunately so many other things want to be done, sorry ;-)

[f1vefour]

Trust me there are other things I could be doing as well ;)

[wlatendresse]

Aren't the lives of geeks great? NEVER getting bored ;-)

[wlatendresse]

Well, nevertheless, the btrfs scrub issue is still here, as on 2015-04-30 with all available updates to date.

[anselal]

OtakuWally, I have the same issue with the free space and it is very annoying that they do not try ti fix it. As of this moment, the bug still exists.

[The-Exnor]

@anselal did you tried stopping Kodi 1st? For me works.

[mkreisl]

@anselal Did you already test the kernel (4.1.x) from the staging area? Would be great to get any feedback :smirk:

[wlatendresse]

Well no, I will happily do that, if someone tell me how and where I get the kernel ASAP.

[CurlyMoo]

http://apt.xbian.org/pool/staging/rpi-wheezy/x/xbian-package-kernel/ or http://apt.xbian.org/pool/staging/rpi2-jessie/l/linux-image-bcm2836/

[wlatendresse]

Thanks, I will try ASAP

[mkreisl]

@CurlyMoo Both of your posted URL's are invalid. Did anybody made any cleanups?

[wlatendresse]

The FIRST one is working for me. However browsing the directory hierarchy I don't find any RPI2 directory, which I NEED unfortunately for my RPI2. Or is there only ONE kernel for both versions of the RPI now? Thanks in advance

[wlatendresse]

Is this the right directory?

http://apt.xbian.org/pool/stable/rpi2-jessie/l/linux-image-bcm2836/

I guess my XBian is still running on wheezy. Do I need to take care when updating? What steps are necessary?

Thanks in advance.

[mkreisl]

Yes, should be. Kernel packages are independent. Download the packages you want and install it with dpkg. What I mostly do before installing a new kernel package is to rename the running /boot/kernel.img and /libmodules/kernelversion so dpkg can not remove it from disk. After installing the new kernel you rename /libmodules/kernelversion.bak to its original name again, so you have in case of a fault always the older running kernel on disk.

And no, there are always different kernel packages necessary for RPi1 and RPi2 (different cpu architectures)

[wlatendresse]

OK, so I am running my XBian now on the latest kernel image 4.1.3+ and will monitor any issues. BTW: Thanks for the help!

Another question: Can XBian be updates to Jessie flawlessly right now? TIA

[wlatendresse]

Well until now I had no problems with kernel 4.1.3 so far, except that my transmission client, which worked with the older kernel, simply will not transfer anything anymore, but that a configuration issue I guess (anyone knows if something tun-device or routing related changed between 3.18.2. and the upper one?). I will do some extensive testing (probably today) later with some massive IO usage ...

[wlatendresse]

I had to add some sysctl options, which has not been necessary with an earlier kernel:

net.core.rmem_max = 4194304
net.core.wmem_max = 1048576

Now transmission works flawlessly again ;-) Still have to check the IO issue ...

[wlatendresse]

Until now the 4.1.3+ kernel is running without any kernel sync issues, however I changed from USB stick to HD for transmission, so I will try some IO stuff on the empty stick ASAP. Are somewhere any ready scripts to use for this?