search

adafruit / Adafruit-Pi-Kernel-o-Matic

https://learn.adafruit.com/raspberry-pi-kernel-o-matic
94 stars 16 forks source link

debhelper Build #7

Closed toddtreece closed 9 years ago

toddtreece commented 9 years ago

This branch is an attempt to properly create the required dependencies needed to install the new raspberrypi-bootloader package without breaking apt or forcing an install. This is as close as we could find to the 'official' way to package new RasPi kernels. Documentation on how to build the required set of closely coupled .deb packages is pretty much non-existent, but this seems to be how they are doing it.

Changes

/me goes outside

brennen commented 9 years ago

This looks solid. Testing now; will report back.

brennen commented 9 years ago

Installs, boots clean on Pi 2, allows for installation of new packages without getting stuck in dependency hell. Dig the way the tarball-plus-install.sh thing worked out. Nice work.

ladyada commented 9 years ago

on Pi 1 after an apt-get update upgrade (so on 3.18.5)

pi@raspberrypi ~/kernels/custom_kernel_1.20150208-1 $ sudo ./install.sh \ Installing custom kernel ** (Reading database ... 82113 files and directories currently installed.) Preparing to replace raspberrypi-bootloader 1.20150130-1 (using raspberrypi-boot loader_1.20150208-1_armhf.deb) ... /bin/sh: 0: Illegal option - dpkg: error processing raspberrypi-bootloader_1.20150208-1_armhf.deb (--install) : subprocess new pre-installation script returned error exit status 2 No diversion 'diversion of /boot/COPYING.linux by rpikernelhack', none removed. No diversion 'diversion of /boot/LICENCE.broadcom by rpikernelhack', none remove d. No diversion 'diversion of /boot/bcm2708-rpi-b-plus.dtb by rpikernelhack', none removed. No diversion 'diversion of /boot/bcm2708-rpi-b.dtb by rpikernelhack', none remov ed. No diversion 'diversion of /boot/bcm2709-rpi-2-b.dtb by rpikernelhack', none rem oved. No diversion 'diversion of /boot/bootcode.bin by rpikernelhack', none removed. No diversion 'diversion of /boot/fixup.dat by rpikernelhack', none removed. No diversion 'diversion of /boot/fixup_cd.dat by rpikernelhack', none removed. No diversion 'diversion of /boot/fixup_x.dat by rpikernelhack', none removed. No diversion 'diversion of /boot/kernel.img by rpikernelhack', none removed. No diversion 'diversion of /boot/kernel7.img by rpikernelhack', none removed. No diversion 'diversion of /boot/start.elf by rpikernelhack', none removed. No diversion 'diversion of /boot/start_cd.elf by rpikernelhack', none removed. No diversion 'diversion of /boot/start_x.elf by rpikernelhack', none removed. No diversion 'diversion of /boot/overlays/README by rpikernelhack', none removed . No diversion 'diversion of /boot/overlays/hifiberry-amp-overlay.dtb by rpikernel hack', none removed. No diversion 'diversion of /boot/overlays/hifiberry-dac-overlay.dtb by rpikernel hack', none removed. No diversion 'diversion of /boot/overlays/hifiberry-dacplus-overlay.dtb by rpike rnelhack', none removed. No diversion 'diversion of /boot/overlays/hifiberry-digi-overlay.dtb by rpikerne lhack', none removed. No diversion 'diversion of /boot/overlays/iqaudio-dac-overlay.dtb by rpikernelha ck', none removed. No diversion 'diversion of /boot/overlays/iqaudio-dacplus-overlay.dtb by rpikern elhack', none removed. No diversion 'diversion of /boot/overlays/lirc-rpi-overlay.dtb by rpikernelhack' , none removed. No diversion 'diversion of /boot/overlays/pcf8523-rtc-overlay.dtb by rpikernelha ck', none removed. No diversion 'diversion of /boot/overlays/pps-gpio-overlay.dtb by rpikernelhack' , none removed. No diversion 'diversion of /boot/overlays/w1-gpio-overlay.dtb by rpikernelhack', none removed. No diversion 'diversion of /boot/overlays/w1-gpio-pullup-overlay.dtb by rpikerne lhack', none removed. Errors were encountered while processing: raspberrypi-bootloader_1.20150208-1_armhf.deb Error completing: dpkg

toddtreece commented 9 years ago

@ladyada weird. i just ran the install again on a pi 1 and it went fine. i see the raspberrypi-bootloader pre-install script died on yours because it looks like /bin/sh didn't know how to deal with -e on your pi. what do you get for sh if you try this?

$ cd /bin
$ ls -l | grep " sh "

I get a symlink to dash:

pi@raspberrypi /bin $ ls -l | grep " sh "
lrwxrwxrwx 1 root root      4 Mar 30  2012 sh -> dash

i've also been testing with a fresh copy of the latest raspbian on both pi v1 and v2, so maybe there's a difference if you are running an older copy. thanks for testing this out @brennen & @ladyada!

ladyada commented 9 years ago

yeah this is bizarre. lrwxrwxrwx 1 root root 4 Mar 29 2012 sh -> dash

and

pi@raspberrypi ~ $ apt-cache policy dash
dash:
Installed: 0.5.7-3
Candidate: 0.5.7-3
Version table:
 *** 0.5.7-3 0
     500 http://mirrordirector.raspbian.org/raspbian/ wheezy/main armhf Packages
    100 /var/lib/dpkg/status

and

 pi@raspberrypi ~ $ dash
$ echo -e "hello\nworld"
-e hello
world

note i have not installed occidentalis/bootstrap - we should probably assume that users don't have that installed.../or/ tell them its required before installing (but better to not require!)

also, by chance do you guys have any personal dotfiles?

brennen commented 9 years ago

by chance do you guys have any personal dotfiles?

Hmm. Definitely not on either of my test images. I'll start with a fresh one though and see if I can reproduce.

ladyada commented 9 years ago

ok i have used this Pi 1 for some development, it could be me. will image a fresh raspbian in a bit

toddtreece commented 9 years ago

@ladyada nope. no dotfiles here, and i think the only thing i changed on the image was manually configuring wifi and installing vim.

i'm not sure what you both are using to image, but using pv with dd recently changed my life. dd is no longer a black box, which is nice.

$ pv 2015-01-31-raspbian.img | sudo dd bs=1m of=/dev/disk2
toddtreece commented 9 years ago

@ladyada & @brennen: i think i just figured it out. i'm pretty sure CRLF struck again. maybe try running dos2unix on the install.sh file from the pi, and then try running it again.

i'll add the install script to .gitattributes. grumble.

ladyada commented 9 years ago

?? even though its tgz'd in vagrant? ok will try soon!

toddtreece commented 9 years ago

@ladyada i'm referring to the wrong script that's breaking. .gitattributes correctly sets line endings for *.sh, which covers install.sh, but the preinst & postinst overrides in the debian folder don't have that file extension so they are missed by .gitattributes and converted to CRLF.

toddtreece commented 9 years ago

the .gitattributes file should now have the proper info after a pull on this branch, but i think the .deb files will need a rebuild since the scripts are packaged with them. i'm glad we caught this bug now.

note to self: if unexplainable weird stuff is happening with shell scripts, it's git on windows doing its auto LF -> CRLF magic.

ladyada commented 9 years ago

sigh ok lemme do this up again :)

brennen commented 9 years ago

Argh. Gah, sorry, I got caught up in it being 70° and sunny out in front of my apartment. Anyway, I think we should have a short bug checklist for these projects that starts with "is it line endings?"

ladyada commented 9 years ago

go outside NERD! :)

ladyada commented 9 years ago

not sure why i/my pi is so cursed...i swapped out power supplies and at least do not get a kernel panic on install. still getting the /bin/sh complaint

pi@raspberrypi:~/custom_kernel_1.20150208-1$ sudo ./install.sh \ Installing custom kernel ** (Reading database ... 76944 files and directories currently installed.) Preparing to replace raspberrypi-bootloader 1.20150130-1 (using raspberrypi-bootloader_1.20150208-1_armhf.deb) ... /bin/sh: 0: Illegal option - dpkg: error processing raspberrypi-bootloader_1.20150208-1_armhf.deb (--install): subprocess new pre-installation script returned error exit status 2

ooooh i understand, the crlf is in the preinst file, yeah i see it there, its not in install.sh

pi@raspberrypi:~/custom_kernel_1.20150208-1$ od -t a preinst
0000000   #   !   /   b   i   n   /   s   h  sp   -   e  cr  nl   F   I

sneaky lil thing...

toddtreece commented 9 years ago

@ladyada i'll mess with the windows testing tomorrow. don't waste your time. i wonder if a fresh clone is needed to fix the existing CRLFs

ladyada commented 9 years ago

todd, its totally a crlf its inside the deb ... but i think maybe a git pull wasnt enough, maybe i have to do a new git checkout (?)

ladyada commented 9 years ago

lemme start a totally fresh setup

toddtreece commented 9 years ago

yeah. it's the pre & post install scripts that are included with the deb here: https://github.com/adafruit/Adafruit-Pi-Kernel-o-Matic/tree/debhelper/debian

i was assuming that git pull & rebuilding the kernel packages would fix any CRLF issues, but maybe git doesn't fix the files it has already converted with a pull. the easy way to check before running another build is to see if those files have CRLFs from your vagrant instance. they should be in /vagrant/debian

ladyada commented 9 years ago

yeah they're gone now

  vagrant@vagrant-ubuntu-precise-32:~$ od -t a /vagrant/scripts/package/DEBIAN/preinst
  0000000   #   !   /   b   i   n   /   s   h  sp   -   e  nl  nl   F   I
  0000020   L   E   S   =   "   b   o   o   t   c   o   d   e   .   b   i

its likely something i did wrong :) will rerun now!

toddtreece commented 9 years ago

it's not you, it's my inability to remember to account for windows & linux line ending sadness. this wouldn't have been an issue, but what seems to be the official repo doesn't yet support kernel7.img, so we need to include these overrides to add support for Pi v2. i have a pull req in so we can remove these from our repo, so that should cut down on errors like this: https://github.com/asb/firmware/pull/2

i just added a pre-commit hook to my local copy of the repo to force me to check .gitattributes when i add a new text file to this repo:

#!/usr/bin/env bash
#
# check for new files, and make sure
# they are handled in .gitattributes

names=`git status --porcelain | grep ^A`

# no files added, continue with commit
if [ "$names" == "" ]; then
  exit
fi

# allow for interaction
exec < /dev/tty

# list the new files
echo "files to be added:"
echo -e $names

# ask me if i'm being an idiot
read -p "are you sure these files are all added to .gitattributes to avoid CRLF hell? (y/n): " -n 1 -r

echo

# abort commit if i'm being an idiot
if [[ $REPLY =~ ^[Yy]$ ]]; then
  exit
else
  exit 1
fi
ladyada commented 9 years ago

ok! of course i forgot to switch branches before vagrant up, so doing it again now... whee

yayyyyy! it works!

pi@pi2 ~ $ uname -a
Linux pi2 3.18.6+ #3 PREEMPT Mon Feb 9 03:48:07 UTC 2015 armv6l GNU/Linux
pi@pi2 ~ $ dmesg | head
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 3.18.6+ (root@vagrant-ubuntu-precise-32) (gcc version 4.7.1 20120402 (prerelease) (crosstool-NG 1.15.2) ) #3 PREEMPT Mon Feb 9 03:48:07 UTC 2015

will try on Pi2

ladyada commented 9 years ago 
pi@pi2 ~ $ uname -a
Linux pi2 3.18.6-v7+ #4 SMP PREEMPT Mon Feb 9 03:57:40 UTC 2015 armv7l GNU/Linux
pi@pi2 ~ $ dmesg | head
[    0.000000] Booting Linux on physical CPU 0xf00
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 3.18.6-v7+ (root@vagrant-ubuntu-precise-32) (gcc version 4.8.3 20140106 (prerelease) (crosstool-NG linaro-1.13.1-4.8-2014.01 - Linaro GCC 2013.11) ) #4 SMP PREEMPT Mon Feb 9 03:57:40 UTC 2015

works for me - pull it! :)

toddtreece commented 9 years ago

nice. thanks!

ladyada commented 9 years ago

thx for helping fix - man this is a really funny hangup whenever using windows+vagrant->linux deploy

i could make you like a CRLF-blinking knuckle ring https://learn.adafruit.com/3d-printed-led-knuckle-jewelry :)

toddtreece commented 9 years ago

hah! i could continue in the tradition of night of the hunter & do the right thing with the battle of CRLF and LF

night of the hunter

thanks for being patient while the battle played out again.