This fixes a subtle and hard-to-reproduce error when building the linux
kernel and modules, and should be The Right Thing To Do in general. The
symptom, when the bug appears, is that the kernel is built with some
version string "4.5.6-01234-g0abcdef", but the modules (both the native
kernel modules and those built externally using the linux-dev package)
end up in /lib/modules/4.5.6-01234-g0abcdef-dirty.
The problem turned out to be that when we do 'git checkout -q', the
files
sometimes ended up with the latter being newer than the former (in the
vast majority of cases, their mtimes are identical due to the limited
resolution of filesystem timestamps), which in turn causes the kernel
build system to (re)generate aesbs-core.S_shipped by running the perl
script.
Now, when we build vmlinux, the kernel build system calls
scripts/setlocalversion rather early to generate
include/config/kernel.release. setlocalversion in turn uses 'git
diff-index HEAD' to decide whether the kernel tree is dirty. At this
point in time it is not, so include/config/kernel.release is create with
the contents "4.5.6-01234-g0abcdef".
Later during the same build, Make detects that the .S_shipped is older
than the perl script, so it gets recreated. Since it is updated with ">"
shell redirection, the dev and ino stays the same, so does the content,
but the mtime is obviously updated.
If we then go on to do a "make LOADADDR=0x1234 uImage", the kernel build
system again calls setlocalversion, and this time "git diff-index
--name-only HEAD" actually produces output (namely,
arch/arm/crypto/aesbs-core.S_shipped), so include/config/kernel.release
is recreated, this time containing "4.5.6-01234-g0abcdef-dirty". The
right version string was already baked into vmlinux, so the resulting
kernel shows that for uname -r, but everything from now on, including
internal and external module builds, ends up using the -dirty string.
When I hit this problem, the time stamps of the files above were 4 ms
apart (which is exactly one jiffy since I have CONFIG_HZ=250, so
apparently that's my filesystem timestamp granularity). This explains
why it's hard to reproduce; a timer tick must hit between git creating
the two files in question (and if it happened to create them in the
other order it would never be seen). Similar problems can happen
anywhere a git repository contains files which are generated by a
Makefile rule, making the build inherently non-deterministic.
So instead of having all the checked-out files having
almost-but-not-quite the same mtime, make them really have the same
time.
However, this now totally wreaks git's view of the world, since all
mtimes and ctimes are now different from what was recorded in the index
during checkout, which in turn means that the very first run of
setlocalversion would treat the kernel tree as dirty. While
deterministic, we don't want all kernel version strings to contain
-dirty, so fix this up by refreshing the index after the mtime-settings.
This fixes a subtle and hard-to-reproduce error when building the linux kernel and modules, and should be The Right Thing To Do in general. The symptom, when the bug appears, is that the kernel is built with some version string "4.5.6-01234-g0abcdef", but the modules (both the native kernel modules and those built externally using the linux-dev package) end up in /lib/modules/4.5.6-01234-g0abcdef-dirty.
The problem turned out to be that when we do 'git checkout -q', the files
arch/arm/crypto/aesbs-core.S_shipped arch/arm/crypto/bsaes-armv7.pl
sometimes ended up with the latter being newer than the former (in the vast majority of cases, their mtimes are identical due to the limited resolution of filesystem timestamps), which in turn causes the kernel build system to (re)generate aesbs-core.S_shipped by running the perl script.
Now, when we build vmlinux, the kernel build system calls scripts/setlocalversion rather early to generate include/config/kernel.release. setlocalversion in turn uses 'git diff-index HEAD' to decide whether the kernel tree is dirty. At this point in time it is not, so include/config/kernel.release is create with the contents "4.5.6-01234-g0abcdef".
Later during the same build, Make detects that the .S_shipped is older than the perl script, so it gets recreated. Since it is updated with ">" shell redirection, the dev and ino stays the same, so does the content, but the mtime is obviously updated.
If we then go on to do a "make LOADADDR=0x1234 uImage", the kernel build system again calls setlocalversion, and this time "git diff-index --name-only HEAD" actually produces output (namely, arch/arm/crypto/aesbs-core.S_shipped), so include/config/kernel.release is recreated, this time containing "4.5.6-01234-g0abcdef-dirty". The right version string was already baked into vmlinux, so the resulting kernel shows that for
uname -r, but everything from now on, including internal and external module builds, ends up using the -dirty string.When I hit this problem, the time stamps of the files above were 4 ms apart (which is exactly one jiffy since I have CONFIG_HZ=250, so apparently that's my filesystem timestamp granularity). This explains why it's hard to reproduce; a timer tick must hit between git creating the two files in question (and if it happened to create them in the other order it would never be seen). Similar problems can happen anywhere a git repository contains files which are generated by a Makefile rule, making the build inherently non-deterministic.
So instead of having all the checked-out files having almost-but-not-quite the same mtime, make them really have the same time.
However, this now totally wreaks git's view of the world, since all mtimes and ctimes are now different from what was recorded in the index during checkout, which in turn means that the very first run of setlocalversion would treat the kernel tree as dirty. While deterministic, we don't want all kernel version strings to contain -dirty, so fix this up by refreshing the index after the mtime-settings.