OS vendor and version autodetection fails on Fedora

rear / rear

Relax-and-Recover - Linux bare metal disaster recovery and system migration solution (cfr. mksysb, ignite)

http://relax-and-recover.org/

GNU General Public License v3.0

921 stars 250 forks source link

OS vendor and version autodetection fails on Fedora #3149

Open lzaoral opened 7 months ago

lzaoral commented 7 months ago

ReaR version ("/usr/sbin/rear -V"): latest HEAD

OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):

$ cat /etc/os-release
NAME="Fedora Linux"
VERSION="39 (Thirty Nine)"
ID=fedora
VERSION_ID=39
VERSION_CODENAME=""
PLATFORM_ID="platform:f39"
PRETTY_NAME="Fedora Linux 39 (Thirty Nine)"
ANSI_COLOR="0;38;2;60;110;180"
LOGO=fedora-logo-icon
CPE_NAME="cpe:/o:fedoraproject:fedora:39"
DEFAULT_HOSTNAME="fedora"
HOME_URL="https://fedoraproject.org/"
DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f39/system-administrators-guide/"
SUPPORT_URL="https://ask.fedoraproject.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=39
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=39
SUPPORT_END=2024-11-12

Description of the issue (ideally so that others can reproduce it):

ReaR does not properly detect OS vendor and version on Fedora:

# /usr/sbin/rear dump
...
# System definition:
  ARCH="Linux-arm"
  OS="GNU/Linux"
  OS_MASTER_VENDOR="Fedora"
  OS_MASTER_VERSION="VERSION_ID=39"
  OS_MASTER_VENDOR_ARCH="Fedora/arm"
  OS_MASTER_VENDOR_VERSION="Fedora/VERSION_ID=39"
  OS_MASTER_VENDOR_VERSION_ARCH="Fedora/VERSION_ID=39/arm"
  OS_VENDOR="RedHatEnterpriseServer"
  OS_VERSION="VERSION_ID=39"
  OS_VENDOR_ARCH="RedHatEnterpriseServer/arm"
  OS_VENDOR_VERSION="RedHatEnterpriseServer/VERSION_ID=39"
  OS_VENDOR_VERSION_ARCH="RedHatEnterpriseServer/VERSION_ID=39/arm"
...

The version should be only 39 and not VERSION_ID=39 and the vendor should be Fedora. Therefore, both the OS_VENDOR_* and OS_MASTER_VENDOR_* should be equal.

cc: @pcahyna

jsmeix commented 7 months ago

As far as I know this variables are set by the function SetOSVendorAndVersion in lib/config-functions.sh

I think this function is rather fragile (some time ago I have had "not so much fun" with it). I assume most of its values are not actually used in ReaR scripts so perhaps things could be simplified to what is actually used?

jsmeix commented 7 months ago

FYI:

What variables are set by SetOSVendorAndVersion:

# grep -o '[A-Z][A-Z_]*=' usr/share/rear/lib/config-functions.sh | cut -d '=' -f1 | sort -u
OS_MASTER_VENDOR
OS_MASTER_VENDOR_ARCH
OS_MASTER_VENDOR_VERSION
OS_MASTER_VENDOR_VERSION_ARCH
OS_MASTER_VERSION
OS_VENDOR
OS_VENDOR_ARCH
OS_VENDOR_VERSION
OS_VENDOR_VERSION_ARCH
OS_VERSION
VERSION_ID

Where those variables are used in ReaR scripts: Here I excluded ReaR scripts that use those variables in a generic way which are

usr/share/rear/lib/config-functions.sh
usr/share/rear/lib/dump-workflow.sh
usr/share/rear/lib/framework-functions.sh
usr/share/rear/lib/validate-workflow.sh
usr/sbin/rear
usr/share/rear/build/default/975_update_os_conf.sh
usr/share/rear/rescue/default/010_merge_skeletons.sh
usr/share/rear/finalize/default/890_finish_checks.sh
usr/share/rear/init/default/005_verify_os_conf.sh

so only those ReaR scripts should be left where the behaviour of ReaR depends on those variables:

# for v in $( grep -o '[A-Z][A-Z_]*=' usr/share/rear/lib/config-functions.sh | cut -d '=' -f1 | sort -u ) ; \
 do echo $v ; \
    find usr/sbin/rear usr/share/rear -type f | xargs grep "$v" | grep -v ': *#' ; \
    echo ======================== ; \
 done \
 | egrep -v 'usr/share/rear/lib/config-functions.sh|usr/share/rear/lib/dump-workflow.sh|usr/share/rear/lib/framework-functions.sh|usr/share/rear/lib/validate-workflow.sh|usr/sbin/rear|usr/share/rear/build/default/975_update_os_conf.sh|usr/share/rear/rescue/default/010_merge_skeletons.sh|usr/share/rear/finalize/default/890_finish_checks.sh|usr/share/rear/init/default/005_verify_os_conf.sh

OS_MASTER_VENDOR
usr/share/rear/layout/prepare/Linux-s390/205_s390_enable_disk.sh:case $OS_MASTER_VENDOR in
usr/share/rear/layout/prepare/Linux-s390/205_s390_enable_disk.sh:        LogPrintError "No code for DASD disk device enablement on $OS_MASTER_VENDOR"
usr/share/rear/rescue/GNU/Linux/310_network_devices.sh:if [[ "$ARCH" == "Linux-s390" && "$OS_MASTER_VENDOR" != "SUSE_LINUX" ]] ; then
usr/share/rear/lib/layout-functions.sh:            case $OS_MASTER_VENDOR in
========================
OS_MASTER_VENDOR_ARCH
========================
OS_MASTER_VENDOR_VERSION
========================
OS_MASTER_VENDOR_VERSION_ARCH
========================
OS_MASTER_VERSION
usr/share/rear/lib/layout-functions.sh:                    if (( $OS_MASTER_VERSION < 12 )) ; then
usr/share/rear/lib/layout-functions.sh:                        if (( $OS_MASTER_VERSION < 7 )) ; then
========================
OS_VENDOR
usr/share/rear/conf/default.conf:OS_VENDOR=generic
usr/share/rear/rescue/GNU/Linux/990_sysreqs.sh:    echo "${OS_VENDOR} ${OS_VERSION}"
usr/share/rear/lib/bootloader-functions.sh:    echo "echo \"build from host: $HOSTNAME ($OS_VENDOR $OS_VERSION $ARCH)\""
usr/share/rear/finalize/Linux-i386/610_EFISTUB_run_efibootmgr.sh:--label ${OS_VENDOR} --loader "$loader" \
usr/share/rear/finalize/Linux-i386/670_run_efibootmgr.sh:    LogPrint "Creating  EFI Boot Manager entry '$OS_VENDOR $OS_VERSION' for '$bootloader' (UEFI_BOOTLOADER='$UEFI_BOOTLOADER') "
usr/share/rear/finalize/Linux-i386/670_run_efibootmgr.sh:    Log efibootmgr --create --gpt --disk $disk --part $partition_number --write-signature --label \"${OS_VENDOR} ${OS_VERSION}\" --loader \"\\${bootloader}\"
usr/share/rear/finalize/Linux-i386/670_run_efibootmgr.sh:    if efibootmgr --create --gpt --disk $disk --part $partition_number --write-signature --label "${OS_VENDOR} ${OS_VERSION}" --loader "\\${bootloader}" ; then
========================
OS_VENDOR_ARCH
========================
OS_VENDOR_VERSION
========================
OS_VENDOR_VERSION_ARCH
========================
OS_VERSION
usr/share/rear/conf/default.conf:OS_VERSION=none
usr/share/rear/rescue/GNU/Linux/990_sysreqs.sh:    echo "${OS_VENDOR} ${OS_VERSION}"
usr/share/rear/lib/bootloader-functions.sh:    echo "echo \"build from host: $HOSTNAME ($OS_VENDOR $OS_VERSION $ARCH)\""
usr/share/rear/finalize/Linux-i386/670_run_efibootmgr.sh:    LogPrint "Creating  EFI Boot Manager entry '$OS_VENDOR $OS_VERSION' for '$bootloader' (UEFI_BOOTLOADER='$UEFI_BOOTLOADER') "
usr/share/rear/finalize/Linux-i386/670_run_efibootmgr.sh:    Log efibootmgr --create --gpt --disk $disk --part $partition_number --write-signature --label \"${OS_VENDOR} ${OS_VERSION}\" --loader \"\\${bootloader}\"
usr/share/rear/finalize/Linux-i386/670_run_efibootmgr.sh:    if efibootmgr --create --gpt --disk $disk --part $partition_number --write-signature --label "${OS_VENDOR} ${OS_VERSION}" --loader "\\${bootloader}" ; then
========================
VERSION_ID
========================

So it seems (according to that quick and dirty test) those variables are not actually used in ReaR scripts

OS_MASTER_VENDOR_ARCH
OS_MASTER_VENDOR_VERSION
OS_MASTER_VENDOR_VERSION_ARCH
OS_VENDOR_ARCH
OS_VENDOR_VERSION
OS_VENDOR_VERSION_ARCH
VERSION_ID

jsmeix commented 7 months ago

What directories we have where the value of such variables is used as directory name (also a quick and dirty test):

I exluded some generic directories and those where the OUTPUT value

# grep -o 'OUTPUT=[A-Z]*' usr/share/rear/conf/default.conf | sort -u

OUTPUT=ISO
OUTPUT=OBDR
OUTPUT=PXE
OUTPUT=RAMDISK
OUTPUT=RAWDISK
OUTPUT=USB

or the BACKUP value

# grep -o 'BACKUP=[A-Z][A-Z0-9]*' usr/share/rear/conf/default.conf | sort -u

BACKUP=AVA
BACKUP=BACULA
BACKUP=BAREOS
BACKUP=BLOCKCLONE
BACKUP=BORG
BACKUP=CDM
BACKUP=DP
BACKUP=DUPLICITY
BACKUP=EXTERNAL
BACKUP=FDRUPSTREAM
BACKUP=GALAXY
BACKUP=GALAXY10
BACKUP=GALAXY11
BACKUP=GALAXY7
BACKUP=NBKDC
BACKUP=NBU
BACKUP=NETFS
BACKUP=NFS4SERVER
BACKUP=NSR
BACKUP=RBME
BACKUP=REQUESTRESTORE
BACKUP=RSYNC
BACKUP=SESAM
BACKUP=TSM
BACKUP=YUM
BACKUP=ZYPPER

and where GNU and Linux is the last directory name

# pushd  usr/share/rear/ ; \
 find . \
 | sed -e 's|/[0-9][0-9][0-9].*.sh||g' \
 | sort -u \
 | egrep -v '/conf/examples|/conf/templates|/lib|/skel|/default|readme|README|shellcheckrc' \
 | egrep -v '/ISO$|/OBDR$|/PXE$|/RAMDISK$|/RAWDISK$|/USB$' \
 | egrep -v '/AVA$|/BACULA$|/BAREOS$|/BLOCKCLONE$|/BORG$|/CDM$|/DP$|/DUPLICITY$|/EXTERNAL$|/FDRUPSTREAM$|/GALAXY$|/GALAXY10$|/GALAXY11$|/GALAXY7$|/NBKDC$|/NBU$|/NETFS$|/NFS4SERVER$|/NSR$|/RBME$|/REQUESTRESTORE$|/RSYNC$|/SESAM$|/TSM$|/YUM$|/ZYPPER$' \
 | egrep -v '/GNU$|/Linux$' ; \
 popd

.
./backup
./build
./build/Debian
./build/Debian/i386
./build/OPALPBA
./build/OPALPBA/Linux-i386
./build/SUSE_LINUX
./check
./conf
./conf/Debian
./conf/Debian/ia64.conf
./conf/GNU/Linux.conf
./conf/Linux-i386.conf
./conf/Linux-ia64.conf
./conf/Linux-ppc64.conf
./conf/Linux-ppc64le.conf
./conf/SUSE_LINUX.conf
./conf/Ubuntu
./conf/Ubuntu.conf
./conf/Ubuntu/7.10.conf
./final-mount
./finalize
./finalize/Debian
./finalize/Debian/i386
./finalize/Debian/ppc64le
./finalize/Fedora
./finalize/Fedora/s390
./finalize/Linux-arm
./finalize/Linux-i386
./finalize/Linux-ia64
./finalize/Linux-ppc64
./finalize/Linux-ppc64le
./finalize/SUSE_LINUX
./finalize/SUSE_LINUX/i386
./finalize/SUSE_LINUX/ppc64
./finalize/SUSE_LINUX/ppc64le
./finalize/SUSE_LINUX/s390
./format
./init
./layout
./layout/compare
./layout/do-mount
./layout/precompare
./layout/prep-for-mount
./layout/prep-for-mount/Linux-s390
./layout/prepare
./layout/prepare/Linux-s390
./layout/recreate
./layout/save
./layout/save/FDRUPSTREAM/Linux-s390
./layout/save/Linux-arm
./output
./output/IPL
./output/IPL/Linux-s390
./output/ISO/Linux-i386
./output/ISO/Linux-ia64
./output/ISO/Linux-ppc64
./output/ISO/Linux-ppc64le
./output/OBDR/Linux-i386
./output/OBDR/Linux-ia64
./output/OBDR/Linux-ppc64
./output/OBDR/Linux-ppc64le
./output/RAWDISK/Linux-i386
./output/USB/Linux-i386
./pack
./prep
./prep/ISO/Linux-i386
./prep/ISO/Linux-ia64
./prep/Linux-arm
./prep/Linux-s390
./prep/OBDR/Linux-i386
./prep/OBDR/Linux-ia64
./prep/OPALPBA
./prep/OPALPBA/Linux-i386
./prep/RAWDISK/Linux-i386
./prep/TAPE
./prep/USB/Linux-arm
./prep/USB/Linux-i386
./prep/USB/Linux-ppc64
./prep/USB/Linux-ppc64le
./prep/USB/Linux-s390
./rescue
./restore
./restore/Fedora
./restore/NETFS/Linux-i386
./restore/OPALPBA
./restore/SUSE_LINUX
./setup
./verify
./wrapup

So it seems only those values are actually used as directory names:

Debian
Fedora
IPL
Linux-arm
Linux-i386
Linux-ia64
Linux-ppc64
Linux-ppc64le
Linux-s390
SUSE_LINUX
Ubuntu
i386
ppc64
ppc64le
s390

By the way: Currently I do not understand how usr/share/rear/output/IPL/Linux-s390/800_create_ipl.sh gets called because I fail to see where the value IPL is set in ReaR.

lzaoral commented 7 months ago

I have also found a related bug. If I create a custom /etc/rear/os.conf with correct values for Fedora, the OS_* and OS_MASTER_* variables will match. Therefore, the scripts array in the SourceStage function will contain some Fedora-specific entries twice because duplicates will not be discarded: https://github.com/rear/rear/blob/096bfde5e234f5a803bae74f24e3821798022c7c/usr/share/rear/lib/framework-functions.sh#L120-L125

For example, the initrd regeneration is then executed twice (rear recover log on s390x):

...
2024-02-06 10:56:46.249979866 ======================
2024-02-06 10:56:46.251775722 Running 'restore' stage
2024-02-06 10:56:46.253627052 ======================
2024-02-06 10:56:46.259645721 Including restore/Fedora/050_copy_dev_files.sh
2024-02-06 10:56:46.270044331 Including restore/Fedora/050_copy_dev_files.sh            <-- DUPLICATE
2024-02-06 10:56:46.278551895 Including restore/default/050_remount_async.sh
2024-02-06 10:56:46.281950834 Including restore/NETFS/default/100_mount_NETFS_path.s
...
2024-02-06 10:57:07.404822080 Including finalize/default/520_confirm_finalize.sh
2024-02-06 10:57:07.408345447 Including finalize/Fedora/550_rebuild_initramfs.sh
2024-02-06 10:57:07.411208165 Original OLD_INITRD_MODULES=(  )
2024-02-06 10:57:07.417129059 New INITRD_MODULES=' dasd_eckd_mod dasd_fba_mod dasd_mod'
2024-02-06 10:57:07.425685641 Running dracut...
2024-02-06 10:57:17.506420149 Updated initrd with new drivers for kernel 6.8.0-0.rc0.20240112git70d201a40823.5.fc40.s390x.
2024-02-06 10:57:17.511338220 Including finalize/Fedora/550_rebuild_initramfs.sh        <-- DUPLICATE
2024-02-06 10:57:17.514697050 Original OLD_INITRD_MODULES=(  )
2024-02-06 10:57:17.519743182 New INITRD_MODULES=' dasd_eckd_mod dasd_fba_mod dasd_mod'
2024-02-06 10:57:17.529820643 Running dracut...
2024-02-06 10:57:26.938005410 Updated initrd with new drivers for kernel 6.8.0-0.rc0.20240112git70d201a40823.5.fc40.s390x.
2024-02-06 10:57:26.944130040 Including finalize/Fedora/s390/660_install_zipl.sh
2024-02-06 10:57:26.946353158 Installing boot loader ZIPL...
2024-02-06 10:57:28.017620673 Including finalize/Fedora/s390/660_install_zipl.sh        <-- DUPLICATE
2024-02-06 10:57:28.021901091 Including finalize/default/880_check_for_mount_by_id.sh
...

lzaoral commented 7 months ago

By the way: Currently I do not understand how usr/share/rear/output/IPL/Linux-s390/800_create_ipl.sh gets called because I fail to see where the value IPL is set in ReaR.

It is a supported OUTPUT value on s390x. It just seems that the documentation in default.conf is missing the OUTPUT=IPL entry.

jsmeix commented 7 months ago

Regarding scripts get called twice when different variables have same value: Ugh! It seems whenever I look at ReaR code what I see is a mess that has grown over longer times :-(

jsmeix commented 7 months ago

Regarding missing "OUTPUT=IPL" documentation in default.conf:

@lzaoral could you please make a pull request with the missing "OUTPUT=IPL" documentation in default.conf because you actually use ReaR on s390x so you know better than me what to tell our users about "OUTPUT=IPL". I do no longer test ReaR on IBM Z and I never managed to boot the ReaR recovery system via the initial program loader on IBM Z 'zipl'. Thank you in advance!

lzaoral commented 6 months ago

@jsmeix Hopefully, I'll get to the documentation of the IPL output option this week. Right now, me and @pcahyna are fixing the packaging of ReaR in Fedora (which we finally co-maintain) in the preparation of the initial branching of CentOS Stream 10.

lzaoral commented 6 months ago

@jsmeix I've found out, that the initial PR with s390x support in ReaR was a bit messy. The IPL option is completely redundant because it does the exact same thing as the already existing and documented RAMDISK option: https://github.com/rear/rear/blob/d48d071ddb6e8d70a42cbc3e6ffec0cb037e1ba1/usr/share/rear/output/IPL/Linux-s390/800_create_ipl.sh#L1-L4

vs.

https://github.com/rear/rear/blob/d48d071ddb6e8d70a42cbc3e6ffec0cb037e1ba1/usr/share/rear/output/RAMDISK/default/900_copy_ramdisk.sh#L15-L23

Therefore, I suggest to just remove the IPL subtree and to add a fallback that will replace OUTPUT=IPL with OUTPUT=RAMDISK during the prep phase to be backwards compatible with existing local.conf files. Unless there are objections, I'll prepare this clean-up PR tomorrow.

jsmeix commented 6 months ago

@lzaoral thank you for having a look at our s390 code!

That IPL is redundant (same as RAMDISK) is what I also thought but I was unsure because I could no longer test ReaR on IBM Z and because your above https://github.com/rear/rear/issues/3149#issuecomment-1935589747 indicated that IPL is actually used.

I fully agree with your proposed clean-up and I look forward to your PR!

By the way regarding documentation about ReaR on IBM Z:

Could you write some initial documentation about "ReaR on IBM Z"?

Only something short so that we have a starting point which then can get enhanced step by step as needed?

In particular I would be much interested in some documentation how to use ReaR on IBM Z because I never tested what @mutable-dan had implemented. But I would like to try that out (as time permits) to get at least an initial basic experience how ReaR can be used on IBM Z.

In particular my main point where I know basically nothing is how to make some "IPL-able thingy" from ReaR's initrd and the kernel and how to actually "IPL" that on a replacement VM on IBM Z. I got lots of IBM documenation also about booting on IBM Z - but that is a so much different world that I gave up trying to understand all that (hundreds of pages of IBM Z domain specific language). So what I would like to get is some real example how to make some real "IPL-able thingy" and how to actually "IPL" that on one specific kind of IBM Z VM.

lzaoral commented 6 months ago

Hmm, the code used to parse the /etc/os-release is quite brittle. Problems I've encountered so far:

The only reason Fedora gets classified as RHEL is this:

$ grep -i -E '(centos|redhat|scientific|oracle)' /etc/os-release
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Fedora"
REDHAT_BUGZILLA_PRODUCT_VERSION=39
REDHAT_SUPPORT_PRODUCT="Fedora"
REDHAT_SUPPORT_PRODUCT_VERSION=39

which is clearly wrong. ~~The NAME="Fedora Linux" value should have been parsed instead.~~ edit: Using ID=fedora is even better.

The VERSION_ID value is parsed incorrectly because there is no " character on Fedora:
```
$ grep "^VERSION_ID=" /etc/os-release | cut -d\" -f2
VERSION_ID=39
```
The expected value is 39. The solution proposed in https://github.com/rear/rear/pull/2950#discussion_r1129753898 would have fixed that.
The generation of the /etc/rear/os.conf file is permanent. Once it is created, it won't be regenerated unless it is manually removed. I have overridden that in the rear Fedora package but ReaR installs from upstream will contain bogus values due to bugs above and/or will contains incorrect information after major version system upgrades (this applies to any other OS).

I don't see a reason to create this file at all because it may do more harm than good. I suggest to just get rid of it as also implied/discussed in https://github.com/rear/rear/issues/1639#issuecomment-361560572 or https://github.com/rear/rear/pull/2954#issuecomment-1467929248).

edit: reformatted

lzaoral commented 6 months ago

While investigating a possible solution for problems above, I've also found two additional problems:

When OS_VENDOR is set to RedHatEnterpriseServer, OS_MASTER_VENDOR is set to Fedora and the OS_MASTER_VERSION is set to the major version for RHEL 5-7 or unchanged for newer releases. This is wrong for at least two reasons:
1. The following version comparison will fail for newer RHELs, e.g. 8.8: https://github.com/rear/rear/blob/9793364ba85aac019f367f45f906e55e2e4d4648/usr/share/rear/lib/layout-functions.sh#L1045-L1064 because:
```
$ bash -c '(( "8.8" < 7 ))'
bash: line 1: ((: 8.8 < 7 : syntax error: invalid arithmetic operator (error token is ".8 < 7 ")
```
2. Since OS_MASTER_VENDOR is Fedora, it would be more sensible to use the actual Fedora release that the given RHEL release branched from (e.g. 34 for RHEL 9).
For all SUSE-relates systems, OS_MASTER_VENDOR is set to SUSE so the following comparison will always be true: https://github.com/rear/rear/blob/9793364ba85aac019f367f45f906e55e2e4d4648/usr/share/rear/rescue/GNU/Linux/310_network_devices.sh#L26

jsmeix commented 6 months ago

@lzaoral while you are at it and if you really have nothing better to do ;-) feel free to mercilessly simplify and clean up that part in ReaR as you think what works best for Red Hat and Fedora. Then I will have a look how far things still work for SUSE and if needed I will make further adaptions for SUSE. But because at least I have something better to do ;-) it is perfectly OK for me when you do only minimal changes to make things work again for Red Hat and Fedora and leave the "great cleanup work" to others - as I do :-)

lzaoral commented 6 months ago

@jsmeix I have already some changes locally that rework the SetOSVendorAndVersion function by properly parsing relevant fields in /etc/os-release. I've just split changes that should be more or less safe into #3165.

jsmeix commented 6 months ago

Regarding OS_VENDOR values:

What OS_VENDOR value dependant scripts we have:

# for OS_VENDOR in Arch RedHatEnterpriseServer Debian Fedora SUSE_LINUX Ubuntu ; \
 do echo $OS_VENDOR ; find usr/sbin/rear usr/share/rear/ | grep "$OS_VENDOR" ; \
    echo ============================================== ; \
 done | grep -v 'usr/share/rear/lib/validated'

Arch
==============================================
RedHatEnterpriseServer
==============================================
Debian
usr/share/rear/conf/Debian
usr/share/rear/conf/Debian/ia64.conf
usr/share/rear/skel/Debian
usr/share/rear/skel/Debian/default.tar.gz
usr/share/rear/finalize/Debian
usr/share/rear/finalize/Debian/ppc64le
usr/share/rear/finalize/Debian/ppc64le/550_rebuild_initramfs.sh
usr/share/rear/finalize/Debian/i386
usr/share/rear/finalize/Debian/i386/550_rebuild_initramfs.sh
usr/share/rear/build/Debian
usr/share/rear/build/Debian/610_jessie_link_systemd_lib.sh
usr/share/rear/build/Debian/i386
usr/share/rear/build/Debian/i386/600_fix_debian_stuff.sh
==============================================
Fedora
usr/share/rear/restore/Fedora
usr/share/rear/restore/Fedora/050_copy_dev_files.sh
usr/share/rear/skel/Fedora
usr/share/rear/skel/Fedora/default
usr/share/rear/skel/Fedora/default/etc
usr/share/rear/skel/Fedora/default/etc/sysconfig
usr/share/rear/skel/Fedora/default/etc/sysconfig/network-scripts
usr/share/rear/skel/Fedora/default/etc/sysconfig/network-scripts/.gitignore
usr/share/rear/skel/Fedora/default/var
usr/share/rear/skel/Fedora/default/var/run
usr/share/rear/skel/Fedora/default/var/run/rpcbind
usr/share/rear/skel/Fedora/default/var/run/rpcbind/.gitignore
usr/share/rear/finalize/Fedora
usr/share/rear/finalize/Fedora/s390
usr/share/rear/finalize/Fedora/s390/660_install_zipl.sh
usr/share/rear/finalize/Fedora/550_rebuild_initramfs.sh
==============================================
SUSE_LINUX
usr/share/rear/conf/SUSE_LINUX.conf
usr/share/rear/restore/SUSE_LINUX
usr/share/rear/restore/SUSE_LINUX/910_create_missing_directories.sh
usr/share/rear/finalize/SUSE_LINUX
usr/share/rear/finalize/SUSE_LINUX/ppc64
usr/share/rear/finalize/SUSE_LINUX/ppc64/550_rebuild_initramfs.sh
usr/share/rear/finalize/SUSE_LINUX/s390
usr/share/rear/finalize/SUSE_LINUX/s390/660_install_grub2_and_zipl.sh
usr/share/rear/finalize/SUSE_LINUX/ppc64le
usr/share/rear/finalize/SUSE_LINUX/ppc64le/550_rebuild_initramfs.sh
usr/share/rear/finalize/SUSE_LINUX/i386
usr/share/rear/finalize/SUSE_LINUX/i386/675_install_shim.sh
usr/share/rear/finalize/SUSE_LINUX/i386/550_rebuild_initramfs.sh
usr/share/rear/build/SUSE_LINUX
usr/share/rear/build/SUSE_LINUX/610_link_systemd_lib.sh
==============================================
Ubuntu
usr/share/rear/conf/Ubuntu
usr/share/rear/conf/Ubuntu/7.10.conf
usr/share/rear/conf/Ubuntu.conf
==============================================

So we neither have scripts for 'Arch' nor for 'RedHatEnterpriseServer'.

In particular regarding 'RedHatEnterpriseServer':

Where 'RedHatEnterpriseServer' appears in our code:

# find usr/sbin/rear usr/share/rear/ -type f \
 | xargs grep 'RedHatEnterpriseServer' \
 | grep -v 'usr/share/rear/lib/validated'

usr/share/rear/rescue/GNU/Linux/990_sysreqs.sh:
    # OS_VENDOR=RedHatEnterpriseServer

usr/share/rear/lib/config-functions.sh:
grep -q -i -E '(centos|redhat|scientific|oracle)' /etc/os-release && OS_VENDOR=RedHatEnterpriseServer
grep -q -i -E '(centos|redhat|scientific|oracle)' /etc/system-release && OS_VENDOR=RedHatEnterpriseServer
OS_VENDOR=RedHatEnterpriseServer

So I think setting 'RedHatEnterpriseServer' can be removed in SetOSVendorAndVersion() in lib/config-functions.sh because 'RedHatEnterpriseServer' is nowhere used.

Regarding where 'Arch' is used in our code:

# find usr/sbin/rear usr/share/rear/ -type f \
 | xargs grep 'Arch' \
 | grep -v 'usr/share/rear/lib/validated' \
 | egrep -v 'Archiv|Architecture'

usr/share/rear/conf/default.conf:
#   /usr/share/kbd/keymaps is used by SUSE and Arch Linux

usr/share/rear/finalize/Fedora/s390/660_install_zipl.sh:
#       | Arch           | Bootloader | Configuration File              |

usr/share/rear/prep/default/320_include_uefi_env.sh:
# Most likely, only recent OSes will be UEFI capable, such as SLES11, RHEL6, Ubuntu 12.10, Fedora 18, Arch Linux

usr/share/rear/prep/GNU/Linux/400_guess_kernel.sh:
    # Arch Linux kernel may not have been found above under /boot
            Log "No Arch Linux kernel file '$KERNEL_FILE' found"

usr/share/rear/rescue/default/100_hostname.sh:
# For Arch Linux storing the host name in /etc/hostname (lowercase)
# put hostname in $ROOTFS_DIR/etc/hostsname (Needed by Arch Linux)

usr/share/rear/rescue/GNU/Linux/500_clone_keyboard_mappings.sh:
#   Arch Linux: /usr/share/kbd/keymaps
#   /usr/share/kbd/keymaps is used by SUSE and Arch Linux

usr/share/rear/lib/config-functions.sh:
            grep -q -i 'arch' /etc/os-release && OS_VENDOR=Arch
            OS_MASTER_VENDOR="Arch"

So I think setting 'Arch' might be removed in SetOSVendorAndVersion() in lib/config-functions.sh because the value 'Arch' is nowhere used in our code. On the other hand detection of Arch Linux via SetOSVendorAndVersion() could be useful in general regardless that currently support for Arch Linux is implemented generically without using the value 'Arch' explicitly in the code.

Because the value 'Arch' is not used I would suggest to introduce a non-ambiguous value for Arch Linux detection in SetOSVendorAndVersion() i.e. 'Arch_Linux' instead of the ambiguous 'Arch' (the latter is the usual abbreviation for 'Architecture').

jsmeix commented 6 months ago

What I think what is needed and what works in practice (in particular what is maintainable in practice) is only two semantically different/separated variables:

one for the Linux distribution
one for the hardware architecture

Nothing more - in particular no versions.

Why no versions Testing for version values is almost always plain wrong because testing for version values is almost always not testing "the real thing" so a version value test could too easily have a wrong result. E.g. assume in version 1 there is /etc/old which is replaced since version 2 by /etc/new then /etc/old versus /etc/new should be directly tested instead of an indirect test of the version that has a wrong result when e.g. /etc/old is still used regardless that version 2 is already there.

So when I have nothing better to do cf. https://github.com/rear/rear/issues/3149#issuecomment-1968483977 I would mercilessly simplify and clean up that part in ReaR to have only two variables left

DISTRIBUTION
ARCHITECTURE

which matches what we actually have as scripts for 'Debian' 'Fedora' and 'SUSE_LINUX' (nothing version dependant there), cf. https://github.com/rear/rear/issues/3149#issuecomment-1972731026

This is known to be sufficient and works in practice since many years.

The only exception is

usr/share/rear/conf/Ubuntu/7.10.conf
usr/share/rear/conf/Ubuntu.conf

where usr/share/rear/conf/Ubuntu/7.10.conf has 'logd' additionally in PROGS which looks solvable generically (i.e. without version ).

lzaoral commented 6 months ago

I agree with your suggestion, @jsmeix! Just for the sake of completeness, OS_MASTER_VERSION is also used in usr/share/rear/lib/layout-functions.sh.

jsmeix commented 6 months ago

@lzaoral when you finished all what you need to do in particular for Fedora and Red Hat (no rush, step by step, take your time) and when that also works for SUSE/openSUSE then (as a totally separated subsequent major step) I will have a look if I could do such a "great cleanup work" (i.e. simplify it to only DISTRIBUTION and ARCHITECTURE) with reasonable effort (provided time permits and provided I have nothing better to do).

github-actions[bot] commented 4 months ago

Stale issue message

github-actions[bot] commented 2 months ago

Stale issue message

gdha commented 3 days ago

@lzaoral milestone is still 'ReaR v2.8' - please adjust as you think it best fits (v3.0 or v3.1). Thanks.

lzaoral commented 3 days ago

@gdha I don't have necessary permissions to do that but https://github.com/rear/rear/pull/3171 which targets v3.0 should fix this issue.