search

raspberrypi / linux

Kernel source tree for Raspberry Pi-provided kernel builds. Issues unrelated to the linux kernel should be posted on the community forum at https://forums.raspberrypi.com/
Other
11.15k stars 5k forks source link

git-revision for released Kernel 3.10.25+ #486

Closed deltaphi closed 10 years ago

deltaphi commented 10 years ago

Using a sequence of dist-upgrade and rpi-update, my Pi running an Raspberry-Foundation Raspian image has arrived at booting Kernel 3.10.25+. I need to build additional kernel modules (namely spi-config and a bunch of CAN-Networking modules). However, I cannot find sources that will let me compile a matching kernel. I have successfully built a Kernel 3.10.24+ that I believe would match the kernel part of the latest raspbian image by extracting the git hash from the firmware. However this is not the Kernel my Pi is running after the dist-upgrade.

Therefore I would like to know, what is the git hash/revision, the current linux-image-3.10.25+ package for raspbian was built from?

Steps To reproduce:

Expected Output: Modules for Kernel version 3.10.25+.

Actual Output: Modules for Kernel version 3.10.25 (note the lacking "+").

popcornmix commented 10 years ago

What does "uname -a" report?

deltaphi commented 10 years ago

pi@raspberrypi ~ $ uname -a Linux raspberrypi 3.10.25+ #622 PREEMPT Fri Jan 3 18:41:00 GMT 2014 armv6l GNU/Linux

In case it is of any concern:

pi@raspberrypi ~ $ cat /etc/issue Raspbian GNU/Linux 7 \n \l

licaon-kter commented 10 years ago

zcat /proc/config.gz > ~/config.rpi-622 and work from that?

deltaphi commented 10 years ago

That is what I have tried so far - see my steps to reproduce. That will give me Kernel 3.10.25 instead of 3.10.25+, so I'm thinking that I am using the wrong source tree.

popcornmix commented 10 years ago

linux-source-3.10 is not the pi specific kernel source, but a generic debian kernel. Try a git clone from here: https://github.com/raspberrypi/linux

(http://elinux.org/RPi_Kernel_Compilation may be useful).

I believe the "+" is appended when you build from a git tree that is not tagged. I think you don't get the "+" as your source didn't come from git.

You can find the firmware repo hash that matches your uname string from here: https://github.com/raspberrypi/firmware/blob/3a4ba0c3964f8f0ecc2dd46246a83258d9e4046d/extra/uname_string

and then pick the corresponding githash: https://github.com/raspberrypi/firmware/blob/3a4ba0c3964f8f0ecc2dd46246a83258d9e4046d/extra/git_hash

which contains: 

16eb921a96db3dc3555a65b53b99c15753e6b770

which is here in linux source repo: https://github.com/raspberrypi/linux/commit/16eb921a96db3dc3555a65b53b99c15753e6b770

deltaphi commented 10 years ago

That appears to be the correct kernel version for my Pi, thank you very much!

I have since figured out that I got this kernel not via dist-upgrade (which is why there is no point in looking for a source package) but via rpi-upgrade. So perhaps telling everyone in every tutorial to first run an rpi-upgrade is not the wisest idea, but oh well...

To help anyone else with the same problem who does not have the very excellent "popcornmix search engine" handy, this is (roughly) how to find the correct git revision - hopefully phrased in a way that helps both experts and newcomers:

Et voila, Kernel Sources.

If anyone knows a better way of searching through all commits of a git repository, let me (and especially stackoverflow) know.

Edit: Now with proper markup.

tz1 commented 10 years ago

Following the above to the letter, and doing the https://github.com/raspberrypi/linux/issues/486 gives me that same long 16...770 hash. Giving me:

$ git checkout 16eb921a96db3dc3555a65b53b99c15753e6b770 fatal: reference is not a tree: 16eb921a96db3dc3555a65b53b99c15753e6b770

It worked when I changed it to --depth=200. If someone else reads this, they may have to increase the depth further.

I don't think the provision of the GPL that insists sources corresponding to binaries meant that they should be buried, hidden, and hard to find among obscure files, hashes, and git commands. Yet it is nearly impossible to rebuild a released kernel image and binaries from sources - because it is nearly impossible to find the right sources in the massive archive. There are git features which could make finding the corresponding source easier. Please consider doing something along those lines.

popcornmix commented 10 years ago

@tz1 please explain how you would like it to be done.

tuffnerdstuff commented 10 years ago

@popcornmix Why don't you use tags? Or would it be possible to provide a linux headers package available via apt-get like other linux distros?

That would make my life so much easier. I was trying to compile a kernel module for one week now ... without luck. I immediately succeeded with arch arm (provides linux headers), but I prefer debian for embedded/server stuff.

kenrestivo commented 10 years ago

@popcornmix: I would like a linux-source-3.10.25+ package which corresponds exactly to the binary that is running. This is the Debian way. It also would be a lot more GPL-ish. Indeed, requiring hours of detective work in order to obtain the source certainly goes against the spirit, possibly even the letter of the GPL. If you want help generating such a package I'd be happy to.

kenrestivo commented 10 years ago

Though, it seems I was able to cheat by doing as advised here, thusly:

sudo ln -s /lib/modules/3.10-3-rpi/build /lib/modules/3.10.25+/build

So my particular itch looks like it is now scratched. Still think a proper source package or at least headers package would be more Debian-ish though, and would be willing to help if needed.

popcornmix commented 10 years ago

@kenrestivo Sure, if you provide a script that produces a suitable kernel headers package, I could keep that up to date. I don't know if a kernel source package makes sense when the code is on github. I could perhaps tag the Linux source tree on each firmware build to make finding the matching source easier. Suggestions of exactly what would be useful are welcome.

tz1 commented 10 years ago

I have various things to speed up finding such in my pikernomatic project. I have the hashes and scripts that fetch the right hash (that is what needs to be tagged), and some docs explaining how to find the "right" hash.

The project does more as it also fetches the tools for a pc build and does some patching for CAN, PiTFT, and a few other things, enabled or disabled in a setup script.

They might provide some base for what you want to do, or simply be helpful.

tz1 commented 10 years ago

That project is at https://github.com/tz1/pikernomatic

kenrestivo commented 10 years ago

That's useful, thanks. Seems like you did a lot of the heavy lifting already.

kenrestivo commented 10 years ago

@popcornmix: a simple git tag (i.e. v3.10.25+) would be awesome, straightforward, easy to do, and probably the clearest, thanks.

tz1 commented 10 years ago

There is a 'tag' function. It would help if there were a tag for each binary release, with either the date or version or anything - there are only a handful of binary releases, when hexxeh or the raspi foundation or someone else produces a binary, having a tag - the version number is probably sufficient, but something like '3.10.33+ cam support" would work. Or even just "binary release" since the date or other metadata is avilable. I don't know how or if the binary releases are coordinated with this tree. They seem to show up, but I found where they try to put the hash in their tree which links to this one.

asb commented 10 years ago

@kenrestivo the machinery to generate a the Debian packages is at github.com/asb/firmware. I would happily accept any patches to improve the situation.

popcornmix commented 10 years ago

How about tagging linux tree with:

git tag -a "`date +%Y%m%d_%H%M%S`" -m "`cat ~/firmware/extra/uname_string`"

With uname_string from https://github.com/raspberrypi/firmware/blob/master/extra/uname_string

notro commented 10 years ago

I see that git_hash and Module.symvers is now present in https://github.com/Hexxeh/rpi-firmware Add .config as well, and then copy those three somewhere now that rpi-update deletes the downloaded files by default.

With that in place, make a kernel source install script like this: https://github.com/notro/rpi-firmware/blob/master/extra/install_kernel_source

Now it's very easy to build kernel modules: https://github.com/notro/fbtft-spindle/wiki/Building-kernel-modules

popcornmix commented 10 years ago

Do you need config? Is this not sufficient: zcat /proc/config.gz

This will avoid deleting git_hash and Module.symver sudo SKIP_REPODELETE=1 rpi-update

notro commented 10 years ago

Yes, I know of those. It was just my input on a way to make it easy for everybody to build kernel modules. Not just the few that can dig up the bits and pieces that are needed.

tz1 commented 10 years ago

The long-form tag would work fine. I might otherwise want something shorter, but it is better to have too much rather than too little.

The /proc/config.gz works, but has to be obtained on the pi itself. I cross compile since I don't want to wait hours. It would be useful to have the .config to avoid the step, but it hasn't been a problem.

Some things (like the PiTFT and other fbtft) need the kernel build since the backlight setting changes and that affects the tree. Also increasing the number of interrupts.

popcornmix commented 10 years ago

We always build the kernel using this config: https://github.com/raspberrypi/linux/blob/rpi-3.12.y/arch/arm/configs/bcmrpi_defconfig so that can be used as well.

kenrestivo commented 10 years ago

Great. For a "script" I'd like to use make-kpkg which is already in Debian (or a thin wrapper around it to fetch the right tag), and with which I'm already familiar, if everyone's OK with that. I hope to have some time tonight to try some builds and see how it goes.

kenrestivo commented 10 years ago

OK, it seems like based on the suggestions above, people are more interested in hand-rolling scripts instead of using the Debian tools. So, in general, before I spend too much time on this, does anyone but me care about making this follow Debian conventions and such as much as possible?

deltaphi commented 10 years ago

I would be interested in a solution "the debian way". It definitely removes the overhead of learning how to use yet another set of tools. Plus, having dpkg packets should ease the pain of juggling multiple kernel versions.

popcornmix commented 10 years ago

I think either a script or a debian package will be more useful than doing it manually. Doing it the debian way would be preferable, so please continue with your investigation.

If you can explain how to run make-kpkg to produce the deb files, we can update that with firmware builds, and get it into @asb's repo so it's available to raspbian users.

kenrestivo commented 10 years ago

@deltaphi: Great. I built a kernel last night using make-kpkg. I'll have to do some more work tweezing the --revision and --append-to-version parameters to get the version naming right, but so far so good. Also I'll need to get it to include the ungzipped kernel in the package, and add some kind of post-install script that'll put it in /boot/kernel.img. Also, I built it using the Raspberry toolchain at https://github.com/raspberrypi/tools . Not sure if that's the most Debian-ish one to use, or if crosstools-ng would be better, etc.

kenrestivo commented 10 years ago

@popcornmix: Will do. As a first pass, I did this, but CAUTION it is not right yet, still have work to do:

     git clone  https://github.com/raspberrypi/linux/
     cd linux/
     git checkout 16eb921a96db3dc3555a65b53b99c15753e6b770
     git checkout  -b v3.10.25+
     export ARCH=arm
     export DEB_HOST_ARCH=armhf
     export CONCURRENCY_LEVEL=6
     export  EXTRAVERSION=-raspbian-20140107
     export CCPREFIX=/mnt/externalsata/raspberry/tools/arm-bcm2708/arm-bcm2708-linux-gnueabi/bin/arm-bcm2708-linux-gnueabi-
     cd /mnt/externalsata/raspberry/kernel/
     wget -O .config  http://where-can-i-get-the-3.10.25-release-config-not-the-3.XXX-config
     export PATH=$PATH:/mnt/externalsata/raspberry/tools/arm-bcm2708/arm-bcm2708-linux-gnueabi/bin
     fakeroot make-kpkg --arch arm --cross-compile arm-bcm2708-linux-gnueabi-  --append-to-version=-release --revision 1 kernel_image kernel_headers kernel_source

Is there a place I can reliably grab the release config (i.e. the 3.10.25+, not an old version, in the case of the 2014-01-07 release) off of the internet without having to scp it from a running Pi? As noted above, I also need to add in the uncompressed kernel to the package, and a post-install script (part of the Debian package) to install it in /boot. I also want to try fakeroot-ng as an alternative to eliminate some spurious error messages.

notro commented 10 years ago

I have made a Linux kernel source installer: http://www.raspberrypi.org/forums/viewtopic.php?f=29&t=76261

bullshit commented 10 years ago

@kenrestivo i created a gist https://gist.github.com/bullshit/8001ce5eb8f03f056199/ @notro :thumbsup: