Compilation failures: Linux bug or TUXML bug?

[FAMILIAR-project]

Hi,

When CONFIG_AIC7XXX_BUILD_FIRMWARE=y there is necessarily a compilation failure. https://cateee.net/lkddb/web-lkddb/AIC7XXX_BUILD_FIRMWARE.html

Can we investigate, using the logs (stdout, stderr, etc.), why this failure occurs?

There are many CIDs to consider. After parsing the CSV file, we can gather all of them with this line: rawtuxdata.query("AIC7XXX_BUILD_FIRMWARE == 'y'")[['cid', 'vmlinux']]

First results on a recent CSV file: 78924 | -1 78948 | -1 78955 | -1 78989 | -1 79002 | -1 79009 | -1 79010 | -1 79037 | -1 79052 | -1

The real question we want to address here is:

• Is it a real bug of the Linux kernel? If yes, can we justify and explain the bug?
• If no, it's a bug of our infrastructure: should we install further packages? What to do for fixing the issue?

(note: there are other compilation failures but let's focus on this specific option at the moment) (note 2: it may be possible that we force CONFIG_AIC7XXX_BUILD_FIRMWARE=n)

[alemasle]

78924 | -1 aicdb.h and others 78948 | -1 aicdb.h 78955 | -1 aicdb.h 78989 | -1 aicdb.h 79002 | -1 aicdb.h 79009 | -1 aicdb.h 79010 | -1 as68k 79037 | -1 aicdb.h 79052 | -1 aicdb.h

That is what appear in the err.log for each of them. These are errors we know for a while but it is strange that they appear this close to each other

[FAMILIAR-project]

Let's have an in-depth analysis of cid=79052 (we can assume the same kind of errors occurs for other configurations)

If we look at stdlog

00:15:50 make -C ./drivers/scsi/aic7xxx/aicasm OUTDIR=/TuxML/linux-4.13.3/drivers/scsi/aic7xxx/aicasm/
00:15:50 *** Install db development libraries
00:15:50 bison -d -b aicasm_gram aicasm_gram.y
00:15:50 mv aicasm_gram.tab.c /TuxML/linux-4.13.3/drivers/scsi/aic7xxx/aicasm//aicasm_gram.c
00:15:50 mv aicasm_gram.tab.h /TuxML/linux-4.13.3/drivers/scsi/aic7xxx/aicasm//aicasm_gram.h
00:15:50 gcc -I/usr/include -I. -I/TuxML/linux-4.13.3/drivers/scsi/aic7xxx/aicasm/ aicasm.c aicasm_symbol.c /TuxML/linux-4.13.3/drivers/scsi/aic7xxx/aicasm//aicasm_gram.c /TuxML/linux-4.13.3/drivers/scsi/aic7xxx/aicasm//aicasm_macro_gram.c /TuxML/linux-4.13.3/drivers/scsi/aic7xxx/aicasm//aicasm_scan.c /TuxML/linux-4.13.3/drivers/scsi/aic7xxx/aicasm//aicasm_macro_scan.c -o /TuxML/linux-4.13.3/drivers/scsi/aic7xxx/aicasm//aicasm -ldb
00:15:50   CC      drivers/rtc/rtc-hym8563.o
00:15:51   CC      drivers/regulator/palmas-regulator.o
00:15:51   CC      drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.o
00:15:51 Makefile:38: recipe for target 'aicasm' failed
00:15:51 drivers/scsi/aic7xxx/Makefile:85: recipe for target 'drivers/scsi/aic7xxx/aicasm/aicasm' failed
00:15:51 scripts/Makefile.build:561: recipe for target 'drivers/scsi/aic7xxx' failed

What's interesting to notice is that bison seems needed (remember?) There is also "Install db development libraries"

Looking at the Kconfig documentation https://cateee.net/lkddb/web-lkddb/AIC7XXX_BUILD_FIRMWARE.html:

This option should only be enabled if you are modifying the firmware source to the aic7xxx driver and wish to have the generated firmware include files updated during a normal kernel build. The assembler for the firmware requires lex and yacc or their equivalents, as well as the db v1 library. You may have to install additional packages or modify the assembler Makefile or the files it includes if your build environment is different than that of the author.

It's worth noticing that they mention lex and yacc. It seems consistent with the use of Bison. https://en.wikipedia.org/wiki/GNU_bison

The most crucial information is that there is a problem with aic7xxx/Makefile. Specifically, looking at the source code:

$(OUTDIR)/aicdb.h:
    @if [ -e "/usr/include/db4/db_185.h" ]; then        \
        echo "#include <db4/db_185.h>" > $@;    \
     elif [ -e "/usr/include/db3/db_185.h" ]; then      \
        echo "#include <db3/db_185.h>" > $@;    \
     elif [ -e "/usr/include/db2/db_185.h" ]; then      \
        echo "#include <db2/db_185.h>" > $@;    \
     elif [ -e "/usr/include/db1/db_185.h" ]; then      \
        echo "#include <db1/db_185.h>" > $@;    \
     elif [ -e "/usr/include/db/db_185.h" ]; then       \
        echo "#include <db/db_185.h>" > $@; \
     elif [ -e "/usr/include/db_185.h" ]; then      \
        echo "#include <db_185.h>" > $@;        \
     else                           \
        echo "*** Install db development libraries";    \
     fi

It means that we don't have installed none of ".h" files in our system (Debian) So maybe the real missing file is _"db185.h" http://manual.freeshell.org/db2/ref/program/convert.html

Now let's look at errlog to gain further insights

crypto/twofish_common.c: In function ‘__twofish_setkey’:
crypto/twofish_common.c:700:1: warning: the frame size of 4992 bytes is larger than 2048 bytes [-Wframe-larger-than=]
 }
 ^
aicasm_symbol.c:48:19: fatal error: aicdb.h: No such file or directory
 #include "aicdb.h"
                   ^
compilation terminated.
aicasm_gram.tab.c: In function ‘yyparse’:
aicasm_gram.tab.c:1628:16: warning: implicit declaration of function ‘yylex’ [-Wimplicit-function-declaration]
aicasm_macro_gram.tab.c: In function ‘mmparse’:
/TuxML/linux-4.13.3/drivers/scsi/aic7xxx/aicasm//aicasm_macro_gram.c:64:25: warning: implicit declaration of function ‘mmlex’ [-Wimplicit-function-declaration]
 #define yylex           mmlex
                         ^
aicasm_macro_gram.tab.c:1199:16: note: in expansion of macro ‘yylex’
aicasm_scan.l: In function ‘yylex’:
aicasm_scan.l:420:6: warning: implicit declaration of function ‘mm_switch_to_buffer’ [-Wimplicit-function-declaration]
      yy_switch_to_buffer(temp_state);
      ^~~~~~~~~~~~~~~~~~~
aicasm_scan.l:421:6: warning: implicit declaration of function ‘mmparse’ [-Wimplicit-function-declaration]
      mm_switch_to_buffer(old_state);
      ^~~~~~~
aicasm_scan.l:424:6: warning: implicit declaration of function ‘mm_delete_buffer’ [-Wimplicit-function-declaration]
      yy_switch_to_buffer(old_state);
      ^~~~~~~~~~~~~~~~
make[4]: *** [aicasm] Error 1
make[3]: *** [drivers/scsi/aic7xxx/aicasm/aicasm] Error 2
make[2]: *** [drivers/scsi/aic7xxx] Error 2
make[1]: *** [drivers/scsi] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [drivers] Error 2

The critical error is related to aicdb.h

fatal error: aicdb.h: No such file or directory
#include "aicdb.h"

There are also warnings related to lex (bison?), but it seems less severe.

So let's focus now on this critical/fatal error. I found a related 8 years-old issue: http://linux-scsi.vger.kernel.narkive.com/J6ldkCzj/patch-aic7xxx-add-a-better-library-dependency-description-when-libdb-is-needed

Berkeley libdb seems needed! Maybe we need to install something like: https://packages.ubuntu.com/fr/trusty/libdb-dev https://packages.debian.org/sid/i386/libdb5.3-dev/filelist

I highly suspect that Berkeley libdb actually contains/installs _"db185.h"

[FAMILIAR-project]

To ease our task of reviewing logs, I've added a simple functionality in bdd2csv

# print err, std, out by connecting to the database 
print_logs_configurations(79052) # cid 

[FAMILIAR-project]

Before fixing the bug (if we can!), I suggest to specialize again tuxml.config and add: CONFIG_AIC7XXX_BUILD_FIRMWARE=n

[FAMILIAR-project]

Same story for another option, please specialize it in tuxml.config CONFIG_AIC79XX_BUILD_FIRMWARE=n

[FAMILIAR-project]

I don't have time to investigate the source of error (I will in the following days) but for sure we should set: CONFIG_WANXL_BUILD_FIRMWARE=n

[alemasle]

All three have been added.

[alemasle]

I close this issue and open an other one made for the propositions of pre-set options for config file

[alemasle]

I reopen it until the correction is made

[alemasle]

I may have found a way to dodge the as68k compiling error: http://murga-linux.com/puppy/viewtopic.php?t=80782

CONFIG_WANXL_BUILD_FIRMWARE: Allows you to rebuild firmware run by the QUICC processor. It requires as68k, ld68k and hexdump programs. You should never need this option, say N.

Maybe if we pre-set "CONFIG_WANXL_BUILD_FIRMWARE=n" we can avoid this kind of errors

[FAMILIAR-project]

Maybe if we pre-set "CONFIG_WANXL_BUILD_FIRMWARE=n" we can avoid this kind of errors

This is what we've done by editing tuxml.config (see above). It seems hard to compile configurations with CONFIG_WANXL_BUILD_FIRMWARE, but we cannot say it's a real bug of Linux

[alemasle]

Yes i did not remember, but it is true that we already fix this. I will test if as68k disappear with this

[FAMILIAR-project]

note: "SELECT * FROM Compilations WHERE cid = 88301 or cid = 79052 ORDER BY cid DESC " (same config, different compilation status since @alemasle manually explores/fixes the dependency package and the image... will be integrated into the real Docker/image/code but we keep the specialization to "no")

[alemasle]

as68k disappeared, i close the issue