search

tytso / e2fsprogs

Ext2/3/4 file system utilities
http://ext4.wiki.kernel.org
381 stars 223 forks source link

rpms do not build (tested on openSUSE Tumbleweed & Rocky 8.10/9.4) #199

Open DetlevCM opened 1 month ago

DetlevCM commented 1 month ago

I guess in a way this is a repeat of bug #33 - maybe there is a way to find a solution?

Context:

We have a lustre file system at work (set up and running). For tests and experimentation (also preparing future updates) we want to experiment with lustre as well as build our own packages. The binary path of make, make install seems to work perfectly fine. However one doesn't necessarily want to build lustre from source on all machines, thus building rpms is of interest. (Weirdly enough this randomly worked once on one VM... I don't have it any more though...)

Hence I'm interested in the make rpm option which presents a headache, as the rpm fails to build... While I started in the Whamcloud fork of e2fsprogs, I replicated this with the code from the git repo here. If a fix can be found (which may also potentially just be better instructions), I believe it should ideally be in the upstream repository. (It is also my understanding that there aren't huge differences between the patched e2fsprogs for lutre and the normal one.)

openSUSE is my personal choice, Rocky linux is a necessity as the lustre server requires it (at least in all formally supported forms). The build and errors are/can be obtained in a kvm virtual machine, so they should be readily reproducible.

So, what did I do, what did I get:

The starting position is a default build, then some tweaks:

./configure
make -j
make fullcheck

openSUSE Tumbleweed (up to date as of 2024_10_16)

-> all good after fullcheck

392 tests succeeded     0 tests failed
Creating test_script...

mke rpm:

RPM build errors:
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/blkid
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/findfs
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/fsck.ext4dev
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/mkfs.ext4dev
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/lib64/libblkid.so.*
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/blkid.8*
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/findfs.8*
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/fsck.ext4dev.8*
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/mkfs.ext4dev.8*
make: *** [Makefile:393: rpm] Error 1

So, lets build some extra libraries (as the binaries aren't built): ./configure --enable-libuuid --enable-libblkid --enable-fsck --enable-verbose-makecmds

make -j -> good, make rpm :

RPM build errors:
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/blkid
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/findfs
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/fsck.ext4dev
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/mkfs.ext4dev
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/lib64/libblkid.so.*
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/blkid.8*
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/findfs.8*
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/fsck.ext4dev.8*
    File not found: /home/detlevcm/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/mkfs.ext4dev.8*
make: *** [Makefile:393: rpm] Error 1

???

-> checking in misc the binary blkid was built... (But not findfs it seems). Also, despite requesting a build of the libraries, the libraries are missing? But the archives seem to be built when looking in the directories... (libblkid.a for example)

Rocky 8.10

393 tests succeeded     1 tests failed
Tests failed: m_rootdir_acl
make[2]: *** [Makefile:401: test_post] Error 1
make[2]: Leaving directory '/root/e2fsprogs/tests'
make[1]: *** [Makefile:406: fullcheck] Error 2
make[1]: Leaving directory '/root/e2fsprogs/tests'
make: *** [Makefile:430: fullcheck-recursive] Error 1

-> rename test to _m_rootdir_acl to skip it, tests now pass.

make rpm

RPM build errors:
    line 14: prereq is deprecated: Prereq: /sbin/ldconfig
    line 39: prereq is deprecated: Prereq: /sbin/install-info
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/blkid
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/findfs
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/fsck.ext4dev
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/mkfs.ext4dev
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/lib64/libblkid.so.*
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/lib64/libuuid.so.*
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/bin/uuidgen
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man1/uuidgen.1*
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/blkid.8*
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/findfs.8*
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/fsck.ext4dev.8*
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/mkfs.ext4dev.8*
make: *** [Makefile:393: rpm] Error 1

Rocky 9.4

393 tests succeeded     1 tests failed
Tests failed: m_rootdir_acl
make[2]: *** [Makefile:401: test_post] Error 1
make[2]: Leaving directory '/root/e2fsprogs/tests'
make[1]: *** [Makefile:406: fullcheck] Error 2
make[1]: Leaving directory '/root/e2fsprogs/tests'
make: *** [Makefile:430: fullcheck-recursive] Error 1

-> rename test to _m_rootdir_acl to skip it, tests now pass.

make rpm

RPM build errors:
    line 14: prereq is deprecated: Prereq: /sbin/ldconfig
    line 39: prereq is deprecated: Prereq: /sbin/install-info
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/blkid
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/findfs
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/fsck.ext4dev
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/sbin/mkfs.ext4dev
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/lib64/libblkid.so.*
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/lib64/libuuid.so.*
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/bin/uuidgen
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man1/uuidgen.1*
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/blkid.8*
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/findfs.8*
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/fsck.ext4dev.8*
    File not found: /root/rpmbuild/BUILDROOT/e2fsprogs-1.47.1-0.x86_64/usr/share/man/man8/mkfs.ext4dev.8*
make: *** [Makefile:393: rpm] Error 1

Conclusion:

As it stands, I think that either the make rpm command is bugged (not sure where and how) or I am missing something with respect to building the rpm files and the readme would benefit from updates that clarify how they can/are built.

DetlevCM commented 1 month ago

A quick update:

I have been butchering the spec file a bit to at least get an rpm out of the code... https://github.com/DetlevCM/e2fsprogs/blob/dcm-hacks/e2fsprogs.spec

-> Specifically for openSUSE, it is possible to see how the files are built in the openSUSE build service and to also build versions locally.

DetlevCM commented 3 weeks ago

So, I have been butchering the spec file in a fork of the repo and I get to build rpm files now... This is by no means a good or perfect configuration, but it builds...

This is a patch file for the .spec file for the configuration I got to... specfile.patch.txt

Maybe this could be used to update the spec file? - Given that it does not build rpms as is, building some may be a step forward? @tytso

tytso commented 3 weeks ago

Thanks! Did you test this on both Rocky and Tumbleweed? And was this with the unmodified upstream e2fsprogs or one with some of the Lustre's out-of-tree patches (which as I recall had a few lustre-specific binaries added?).

What I've been thinking about doing is moving the spec file into contrib, with specific spec files for Fedora, RhEL, SLES, Tumbleweed, etc., since in the past I believe that creating a spec file that works on all RPM-based distros can be challenging. Thsi is especially true if we want to make sure that the generated RPM package is a drop-in replacement for the e2fsprogs RPM built by the distro, both in terms of any distro-specific patches that may have been applied, or distros-ecific pathnamaes which might be different.

DetlevCM commented 3 weeks ago

Short answer:

Building on Tumbleweed - using the original source, comparing file paths in openSUSE & Rocky rpms to update the spec file.

Now for the longer answer:

For me, the ideal place to improve things is "upstream". Invariably there will be some distro-specific tweaks (just thinking of the hardening patches in SUSE, whatever they do...) - So I started with the original e2fsprogs code from your repository. (No lustre patches or similar.)

My first objective was to make the spec file work - somehow. This was step one when I posted. Next started to look at the rpm packages in openSUSE Tumbleweed (my "daily driver") and Rocky 9.4 to update the structure (the recent update). I've built the lustre patched version on Rocky - I didn't try building your original source on Rocky. All compilation and testing was done on openSUSE Tumbleweed.

What complicated matters a little bit, is that the distributions have broken down the whole project into individual rpms, which means looking into the individual rpms and figuring our which binary/library/header goes where... Fortunately they are largely similar. One library package has a different name and Rocky has an extra header and binary (fuse) compared to opneSUSE. - These are possibly distro specific problems/configurations that have no easy solution, but if it reduces the number of "special configurations", all the better.

There are also a number of binaries and libraries that I haven't found a home for yet (in the exclude list). The initd rules are old... So there is work to do still, but I've tried taking some small steps in the direction of updating the spec file.

With respect to building and distro specific configurations: There are rpms that work everywhere and there are rpms that are distro specific. It really depends. As this touches on fairly low level stuff, I'm not sure I'm well placed to comment on all the intricacies of small differences in certain paths between distributions. Though openSUSE and Rocky seem to have become fairly similar - at least for this package. Software and library versions may be the bigger differences.

In this respect, moving the spec file to a dedicated directory with version specific adjustments could be a good idea. Simultaneously, given that the distributions patch the code anyway, maybe it makes more sense to have a general good definitions (of the "make install" type) and then let the distributions add their own tweaks as required (which would be less work for you). The downside is that in this case if anybody changes paths, one could end up with two versions of the same executable/library/header...

And what I learnt of the rpmbuild process is that effectively all it seems to do is pack the binaries and headers into an (rpm) archive from a predefined buildroot where it is installed using make install. -> Any distro that thus wants to change paths, would possibly already need to start adjusting the Makefile before then employing a distro-specific spec file.