scsi/sg.h is missing during busybox compilation

[James-E-A]

I'm getting the following error when trying to compile the main userland tarball (full log):

  CC      util-linux/eject.o
util-linux/eject.c:49:22: fatal error: scsi/sg.h: No such file or directory
compilation terminated.
make[2]: *** [util-linux/eject.o] Error 1
make[1]: *** [util-linux] Error 2
make[1]: Leaving directory `/root/bedrocklinux-userland/src/busybox'
make: *** [build/bin/busybox] Error 2

I'm not sure if this is a consequence of the workaround I did earlier, which was to touch src/fuse/config.rpath to fix this error:

Cloning into 'src/fuse'...
touch src/fuse/.success_retreiving_source
mkdir -p /root/bedrocklinux-userland/build
cd src/fuse/ && \
    ./makeconf.sh && \
    ./configure --prefix=/root/bedrocklinux-userland/build --disable-shared --enable-static --disable-util --disable-example && \
    make CC=/root/bedrocklinux-userland/build/bin/musl-gcc && \
    make install
Running libtoolize...
libtoolize: putting auxiliary files in `.'.
libtoolize: copying file `./ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
config.rpath not found! - is gettext installed?
make: *** [build/lib/libfuse.a] Error 1

I don't know where I need to get this file from, though:

find / -name sg.h
/usr/include/scsi/sg.h
/root/bedrocklinux-userland/src/musl/include/scsi/sg.h
/root/bedrocklinux-userland/src/linux_headers/include/scsi/sg.h

These files are 11700 bytes, 3085 bytes, and 12420 bytes, respectively; so I'm not sure which one I should use if I were to just "drop one in". It seems that busybox doesn't come with its own copy of the file.

[James-E-A]

It just kept requesting more and more headers every time I gave it the ones it was looking for..

ln -vs ../../linux_headers/include/{scsi,linux,uapi,asm-generic} src/busybox/include/
ln -vs ../../linux_headers/include/asm-generic src/busybox/include/asm

..at which point it switched from "nice, simple" errors like the one in the OP to this behemoth, and I realized that I would probably no longer get away with just cramming files that "looked about right" into the source tree.

[James-E-A]

Aha,

looking at this mailing list post, it seems that busybox is built against various headers from the libc, in this case that would be musl I suppose

instead, the issue is cured by

ln -vsf ../../musl/include/scsi src/busybox/include/

Leaving open for now, though, since this "fix" is neither apparent nor documented (nor, I suspect, "how it's supposed to be done")

[paradigm]

config.rpath is normally provided by something along the lines of a gettext package. Both the Bedrock Linux installation instructions and the error message you ran into point in this direction, but neither is quite perfect for you because apparently Red Hat distros break this out into a separate gettext-devel package. Try installing gettext-devel and see if you have get a /usr/share/gettext/config.rpath file. If so, remove the local, empty config.rpath. I made a note to explicitly mention gettext-devel in the installation instructions when I get around to updating them, probably this weekend.

You're right that the proper place to get headers for busybox et al is either from linux_headers or musl - that's exactly what linux_headers and musl are for. The corresponding make recipes "install" them into build/, which should result in build/include/scsi/sg.h. The compiler is told to look for headers in build/include (and libraries in build/lib).

Looking at the log you provided, I don't see musl being built/installed. For example, I don't see this line show up in there. I see fuse being compiled in your log despite it being dependent on musl being built/installed first. It's possible there's a issue with the Makefile. Maybe I typo'ed a dependency somewhere. I can't see this as being related to config.rpath.

I do most of my testing with a straight make, no -j flag. If you're using -j, maybe that's triggering an issue in the Makefile that's benign without the flag.

I also do most of my testing straight through from a fresh copy of the userland repo to the resulting tarball. I've not spent any effort testing resumption of aborted attempts. In theory if the Makefile is well written that should just work, but if I've got a mistake somewhere in there breaking on the config.rpath issue then resuming could have triggered an issue that doesn't appear in a straight run.

Per another issue I know you're on an Atom box and compiling can be painful, but my proposal here is to get gettext-devel then try again with a fresh everything and no -j flag.

Even if that fixes the issue, we can leave this open until I find the underlying problem with the Makefile (and add a note about gettext-devel).

[paradigm]

I ran:

sudo mv /usr/share/gettext/config.rpath{,~}
make
# errors about config.rpath
sudo mv /usr/share/gettext/config.rpath{~,}
make
# errors about scsi/sg.h

Thus reproducing your issue exactly. It seems the problem is with the Makefile not resuming an aborted operation correctly. Being able to reproduce this should help me narrow down what is going on. For the time being, I think if you grab a fresh bedrocklinux-userland and try again with gettext-devel it should work for you. Assuming that works for you, I'll poke at the Makefile until I figure out why it's not resuming as expected.

[paradigm]

I figured it out. There were, in fact, multiple issues with telling make about dependencies. For example:

linux_headers: source_linux_headers build/.success_build_linux_headers

build/.success_build_linux_headers:
    mkdir -p build/include
    cd src/linux_headers/ && \
        make headers_install INSTALL_HDR_PATH=$(BUILD)

I'm using build/.success_build_linux_headers to track whether linux_headers has build successfully. However, I never actually touch the file as intended when linux_headers finishes building, and thus make builds linux_headers every time. When doing an undesired call to linux_headers (e.g. after fixing config.rpath), after musl has been built, it runs:

mkdir -p build/include
cd src/linux_headers/ && \
    make headers_install INSTALL_HDR_PATH=/dev/shm/bedrocklinux-userland/build
make[1]: Entering directory '/dev/shm/bedrocklinux-userland/src/linux_headers'
  CHK     include/generated/uapi/linux/version.h
  REMOVE  sg.h scsi_ioctl.h scsi.h
make[1]: Leaving directory '/dev/shm/bedrocklinux-userland/src/linux_headers'

removing sg.h, causing the problem.

The fix is simply touching build/.success_build_linux_headers at the end of the recipe which, presumably, was the intent when I originally wrote that recipe.

There were other, similar issues I found when investigating this one.

I want to review the needed changes with fresh, unrushed eyes over the weekend before committing anything so I don't introduce such silly mistakes again.

[James-E-A]

"On the surface", this seems to be fixed (well, at least, it compiles the tarball, successfully) by just installing the gettext-devel package.

(I can send log if you want, but it's ~1.76MiB)

But since it looks like it was a symptom of an underlying to-be-fixed problem, I'm leaving this issue open, if you want to use it as the tag for the problem.

[paradigm]

I've reworked the Makefile in 286b7732b2c3f1f78bd4eb9bbbb3967904ebbcaa. It should now handle parallelization via -j and resuming interruption correctly.

Some quick and dirty benchmarks, for kicks:

• The previous Makefile, in serial, grabbing upstream sources:

real    4m19.476s
user    1m31.280s
sys     0m9.696s

• The new Makefile, with -j4, grabbing upstream sources:

real    3m27.727s
user    1m32.460s
sys     0m10.104s

• The previous Makefile, in serial, using preexisting upstream sources:

real    2m16.925s
user    1m18.876s
sys     0m6.760s

• The new Makefile, with -j4, using preexisting upstream sources:

real    1m52.883s
user    1m19.292s
sys     0m6.396s

Parallelizing downloads seems to make a noticeable difference. However, once all the sources are on disk, most of the time is consumed by building the linux headers, musl, and busybox, all of which must happen in series anyways, so this isn't a huge jump. I could probably push this further by propagating the job count down to the make calls for components like musl and busybox, but that's out of the scope of this issue, which was just fixing the dependency issues.

[James-E-A]

I don't think reworking the parallelization was necessary, at least for this issue; I had the same error before adding the -j flag as after; I just wanted to expedite the gathering of full logs

(But that's always good to have parallelization working more efficiently, so not complaining! Glad this issue inspired improvements)

[paradigm]

Oh, I didn't explicitly go out to add support for parallel builds. The way make handles resuming interruptions and parallelization have the same requirements. Making parallelization work was a free benefit from fixing making resumption of interrupted runs. That having been said, I should have had that working day 1.