commit 532 => no more kernel-modules in my images

[GoogleCodeExporter]

hi,I did a very long git bisect(I had to rebuild from scratch each time) 
starting from org.openembedded.org and ending up with opkg revision.

The test I was doing was:
rm the whole tmpdir
bitbake bisect-image
tar tvjpf bisect-image-bug.tar.bz2 | grep "lib/modules" #note that bug is a 
machine

prior to commit 532(so commit 531) the modules were shipped in the image,and 
starting at commit 532,they stopped beeing shipped:
$ tar tvjpf bisect-image-bug.tar.bz2 | grep "lib/modules"
tar: Record size = 8 blocks
drwxr-xr-x root/root         0 2010-06-24 08:19 ./lib/modules/
drwxr-xr-x root/root         0 2010-06-24 11:19 ./lib/modules/2.6.27.2/
-rw-r--r-- root/root        81 2010-06-24 11:19 
./lib/modules/2.6.27.2/modules.isapnpmap
-rw-r--r-- root/root        45 2010-06-24 11:19 
./lib/modules/2.6.27.2/modules.alias
-rw-r--r-- root/root        99 2010-06-24 11:19 
./lib/modules/2.6.27.2/modules.pcimap
-rw-r--r-- root/root       626 2010-06-24 11:19 
./lib/modules/2.6.27.2/modules.symbols
-rw-r--r-- root/root        69 2010-06-24 11:19 
./lib/modules/2.6.27.2/modules.ccwmap
drwxr-xr-x root/root         0 2010-06-24 08:19 ./lib/modules/2.6.27.2/kernel/
drwxr-xr-x root/root         0 2010-06-24 11:19 
./lib/modules/2.6.27.2/kernel/fs/
drwxr-xr-x root/root         0 2010-06-24 11:19 
./lib/modules/2.6.27.2/kernel/fs/vfat/
-rw-r--r-- root/root     16728 2010-06-24 08:19 
./lib/modules/2.6.27.2/kernel/fs/vfat/vfat.ko
drwxr-xr-x root/root         0 2010-06-24 11:19 
./lib/modules/2.6.27.2/kernel/fs/fat/
-rw-r--r-- root/root     65548 2010-06-24 08:19 
./lib/modules/2.6.27.2/kernel/fs/fat/fat.ko
-rw-r--r-- root/root        73 2010-06-24 11:19 
./lib/modules/2.6.27.2/modules.ieee1394map
-rw-r--r-- root/root        74 2010-06-24 11:19 
./lib/modules/2.6.27.2/modules.ofmap
-rw-r--r-- root/root       189 2010-06-24 11:19 
./lib/modules/2.6.27.2/modules.usbmap
-rw-r--r-- root/root        43 2010-06-24 11:19 
./lib/modules/2.6.27.2/modules.seriomap
-rw-r--r-- root/root       133 2010-06-24 11:19 
./lib/modules/2.6.27.2/modules.dep
-rw-r--r-- root/root       141 2010-06-24 11:19 
./lib/modules/2.6.27.2/modules.inputmap
here there is only one module.

The modules are included by the following mecanism:
they are defined here:
MACHINE_EXTRA_RRECOMMENDS = "marvell-gspi-fw marvell-sdio-fw kernel-modules"
and included by the task-base recipe in the task-base-machine package:
RRECOMMENDS_task-machine-base = "${MACHINE_EXTRA_RRECOMMENDS}"
the task-base package depends on task-base-machine
RDEPENDS_task-base = "\
[...]
    task-machine-base \
[...]

And finally my bisect-image depends on task-base:

IMAGE_PREPROCESS_COMMAND = "create_etc_timestamp"

IMAGE_INSTALL = "\
    task-base \
"

export IMAGE_BASENAME = "${PN}"
IMAGE_LINGUAS = ""

inherit image

I'll investigate further

Original issue reported on code.google.com by GNUtoo@no-log.org on 24 Jun 2010 at 12:49

[GoogleCodeExporter]

in the case of a bad image I have:
cat installed-packages.txt | grep kernel
kernel_2.6.27.2+svnr10746-r32.5_bug.ipk
kernel-2.6.27.2_2.6.27.2+svnr10746-r32.5_bug.ipk
kernel-image-2.6.27.2_2.6.27.2+svnr10746-r32.5_bug.ipk
kernel-module-fat_2.6.27.2+svnr10746-r32.5_bug.ipk
kernel-module-vfat_2.6.27.2+svnr10746-r32.5_bug.ipk

in the case of a good one I have kernel-modules and all the modules(for 
instance the kernel-module-bmi-lcd-core which is the display/framebuffer driver)

Original comment by GNUtoo@no-log.org on 24 Jun 2010 at 12:52

[GoogleCodeExporter]

strangely I've one of the marvell firmwares:
cat installed-packages.txt | grep marv
marvell-gspi-fw_9.70.3-p37-r0.5_all.ipk

Original comment by GNUtoo@no-log.org on 24 Jun 2010 at 12:53

[GoogleCodeExporter]

Are you saying you bisected back to this commit?
http://cgit.openembedded.org/cgit.cgi/openembedded/commit/?id=731e6defe6da58bff4
82f33508058544ad1013a9

You have inferred that you tried r532; Its probably not advisable to use 
r532-r535 as they are components of a larger fix for Issue #50. However, I 
can't see how the change in r532 would cause different packages to be (not) 
installed unless parsing of possibly different opkg output is now causing 
variation.

If you can track this back to a particular opkg command that has changed 
behaviour and/or output which is causing changed behaviour in OE, then I should 
be able to address this.

Posting build logs for the last known good build and the first bad build may 
help.

Original comment by graham.g...@gmail.com on 25 Jun 2010 at 12:25

[GoogleCodeExporter]

Based upon your failing do_rootfs log http://pastebin.com/j8z1PQia I see the 
following collected errors:

Collected errors:
 * check_data_file_clashes: Package marvell-sdio-fw wants to install file /home/gnutoo/embedded/oe/oetmps/bug/rootfs/bisect-image/lib/firmware/Marvell-Licence.txt
        But that file is already provided by package  * <no package>
Please move this file out of the way and try again.
 * opkg_install_cmd: Cannot install package task-base.
 * resolve_conffiles: Existing conffile /home/gnutoo/embedded/oe/oetmps/bug/rootfs/bisect-image/etc/device_table is different from the conffile in the new package. The new conffile will be placed at /home/gnutoo/embedded/oe/oetmps/bug/rootfs/bisect-image/etc/device_table-opkg.

So it looks like the Marvell-Licence.txt file already exists. Its probably 
being provided by another package, my guess is marvell-gspi-fw (other changes 
between r533 and r536 should fix this message to tell you which package). This 
clash was previously unreported by opkg due to a bug when using offline roots.

The fix is twofold:
(1) OpenEmbedded should check for this condition in 
rootfs_ipk.bbclass:rootfs_ipk_log_check().
(2) The Marvell firmware recipe(s) should be altered to not include this file 
in multiple recipes (add a marvell-fw-license recipe for the others to depend 
upon?).

Original comment by graham.g...@gmail.com on 25 Jun 2010 at 4:38

[GoogleCodeExporter]

I've attached the good and the bad log
I've no ugly log yet(joke)
Denis.

Original comment by GNUtoo@no-log.org on 25 Jun 2010 at 6:12

Attachments:

• log.do_rootfs.9235
• log.do_rootfs.14320

[GoogleCodeExporter]

This is not really a bug in opkg. Its a bug in the marvell OE recipes which 
previously went unnoticed and was exposed by r532. It was difficult to track 
down due to improper error propagation in opkg, which was fixed in r540 - this 
type of data file duplication should now error out in an obvious way.

Original comment by graham.g...@gmail.com on 18 Aug 2010 at 6:01

• Changed state: Fixed