search

rear / rear

Relax-and-Recover - Linux bare metal disaster recovery and system migration solution (cfr. mksysb, ignite)
http://relax-and-recover.org/
GNU General Public License v3.0
940 stars 254 forks source link

Clean up not needed things in predefined LIBS entries #2743

Open jsmeix opened 2 years ago

jsmeix commented 2 years ago

This is a forwarded issue that was initially reported by a SUSE customer and analyzed by a SUSE colleague - here excerpts of what he wrote:

# rear -v mkrescue
...
Copying files and directories
Copying binaries and libraries
Copying kernel modules
Copying all files in /lib*/firmware/
There are binaries or libraries in the ReaR recovery system that need additional libraries
/usr/lib64/syslog-ng/loggen/libloggen_socket_plugin.so requires additional libraries
        libloggen_helper-3.30.so.0 => not found
        libloggen_plugin-3.30.so.0 => not found
/usr/lib64/syslog-ng/loggen/libloggen_ssl_plugin.so requires additional libraries
        libloggen_helper-3.30.so.0 => not found
        libloggen_plugin-3.30.so.0 => not found
ReaR recovery system in '/tmp/rear.1UqzM3iYX37WjeH/rootfs' needs additional libraries, check /var/log/rear/rear-sle15sp3-1.log for details

Specify in etc/rear/local.conf

LIBS+=( /usr/lib64/libloggen_plugin-3.30.so.0 /usr/lib64/libloggen_helper-3.30.so.0 )

usr/share/rear/conf/GNU/Linux.conf holds a line 

/usr/lib*/syslog-ng/*

which makes rear include /usr/lib64/syslog-ng/loggen directory containing some libs for the loggen binary which seems to be just for testing syslog-ng and doesn't need to be in the rear rescue system.

Changing said line to

/usr/lib*/syslog-ng/*so

makes it similar to what rsyslog line looks and solves the issue for me.

Cf. the current code at https://github.com/rear/rear/blob/master/usr/share/rear/conf/GNU/Linux.conf#L182

jsmeix commented 2 years ago

When looking at the LIBS settings in usr/share/rear/conf/GNU/Linux.conf I wonder why there are (so many) predefined LIBS elements needed at all.

Reasoning (as far as I see it):

In usr/share/rear/build/GNU/Linux/390_copy_binaries_libraries.sh the "Copy libraries" code part that is currently at https://github.com/rear/rear/blob/master/usr/share/rear/build/GNU/Linux/390_copy_binaries_libraries.sh#L79 is

local all_libs=( "${LIBS[@]}" $( RequiredSharedObjects "${all_binaries[@]}" "${LIBS[@]}" ) )

This copies all libraries in LIBS plus all libraries for all binaries (i.e. executable programs) and all libraries for all libraries in LIBS that are found by the RequiredSharedObjects function.

It does not find recursively libraries for libraries. This is intentional because the idea behind is to keep the recovery system small. In the end only libraries that are needed by executable programs and libraries that are needed by libraries in LIBS are automatically added to the recovery system.

The RequiredSharedObjects function is in usr/share/rear/lib/linux-functions.sh currently at https://github.com/rear/rear/blob/master/usr/share/rear/lib/linux-functions.sh#L109

Because 'ldd' outputs also transitively required libraries all required libraries by an executable program and all required libraries by a library in LIBS get automatically added to the recovery system but not more.

This way in particular all executable programs should work inside the recovery system provided they are "normally" linked with the libraries that they need.

If an executable program loads a library via dlopen that additional library must be specified in LIBS and then all libraries that are "normally" linked with that additional library would be also automatically added to the recovery system.

Conclusion (as far as I understand it):

Only libraries that are loaded by executable programs via dlopen should be needed to be explicitly listed in LIBS because all other libraries (i.e. "normally" linked libraries) should get automatically included in the recovery sytem as described above.

jsmeix commented 2 years ago

Because of my recent https://github.com/rear/rear/issues/2743#issuecomment-1018280042 I asked my SUSE colleague:

Could you test what happens when you remove the line /usr/lib/syslog-ng/ from /usr/share/rear/conf/GNU/Linux.conf

When that works could you then also test what happens when you add the loggen binary in etc/rear/local.conf via REQUIRED_PROGS+=( loggen )

Here excerpts of what he replied:

Could you test what happens when you remove the line /usr/lib/syslog-ng/ from /usr/share/rear/conf/GNU/Linux.conf

Works just fine, libsyslog-ng is still copied over, but no loggen related libs

When that works could you then also test what happens when you add the loggen binary in etc/rear/local.conf via REQUIRED_PROGS+=( loggen )

Ok, here we go:

sle15sp3-1:/tmp # grep -v ^# /etc/rear/local.conf 
REQUIRED_PROGS+=( loggen )

sle15sp3-1:/tmp # chroot rear.FbeTCDwopdJqWU2/rootfs/
bash-4.4# loggen
error [loggen.c:enumerate_plugins:167] unable to open plugin directory /usr/lib64/syslog-ng/loggen (err=No such file or directory)

And what I conclude from his reply is:

bash-4.4# loggen
error [loggen.c:enumerate_plugins:167] unable to open plugin directory /usr/lib64/syslog-ng/loggen (err=No such file or directory)

This proves that loggen works inside the recovery system because loggen is able to run (no abort during its startup because of missing libraries or other things like that) and it is loggen itself that reports the error so loggen is running and it "correctly" errors out.

Of course with only REQUIRED_PROGS+=( loggen ) one only get that plain binary in the recovery system plus automatically all its "normally" linked libraries. But nothing more.

There is no magic in ReaR that would know loggen needs additional files to run successfully.

To tell ReaR what additional files loggen needs one may use something like COPY_AS_IS+=( /usr/lib64/syslog-ng/loggen )

pcahyna commented 2 years ago

If an executable program loads a library via dlopen that additional library must be specified in LIBS and then all libraries that are "normally" linked with that additional library would be also automatically added to the recovery system.

Hi @jsmeix , I don't understand the reason for the error then. If all libraries that are "normally" linked with that additional library are automatically added to the recovery system, why does the check say that "/usr/lib64/syslog-ng/loggen/libloggen_socket_plugin.so requires additional libraries" ? Since /usr/lib*/syslog-ng/* is specified in LIBS, the reasoning above should have applied to it and the required libraries for /usr/lib64/syslog-ng/loggen/libloggen_socket_plugin.so should have been added automatically.

jsmeix commented 2 years ago

@pcahyna thank you so much for your cutting glance - it helped a lot!

I think I know it: What the code in build/GNU/Linux/390_copy_binaries_libraries.sh basically does is like (I use /usr/lib/qt5/ as example here):

# LIBS+=( /usr/lib*/qt5/* )

# qt5_libs=( "${LIBS[@]}" )

# for e in "${qt5_libs[@]}" ; do echo $e ; done
/usr/lib64/qt5/bin
/usr/lib64/qt5/libexec
/usr/lib64/qt5/plugins
/usr/lib64/qt5/qml

# echo /usr/lib*/qt5/*
/usr/lib64/qt5/bin /usr/lib64/qt5/libexec /usr/lib64/qt5/plugins /usr/lib64/qt5/qml

# find /usr/lib64/qt5/ -type f | wc -l
192

so /usr/lib*/qt5/* does not recursively evaluate to all what is below /usr/lib64/qt5 but only to what is directly in /usr/lib64/qt5

I think the code in build/GNU/Linux/390_copy_binaries_libraries.sh needs to be enhanced for the elements in LIBS similar as it currently builds the all_binaries array from the elements in PROGS and REQUIRED_PROGS so that the elements in LIBS get fully evaluated to all files.

jsmeix commented 2 years ago

With https://github.com/rear/rear/pull/2744 merged the initial specific part of this issue should be fixed.

What is not fixed are the two generic parts of this issue the generic cleanup part as described in https://github.com/rear/rear/issues/2743#issuecomment-1018280042 and the generic bug as described in https://github.com/rear/rear/issues/2743#issuecomment-1022133392

jsmeix commented 2 years ago

ReaR 2.7 should not be released without the generic bug part of this issue fixed as described in https://github.com/rear/rear/issues/2743#issuecomment-1022133392

But ReaR 2.7 can be released without the generic cleanup part of this issue fixed as described in https://github.com/rear/rear/issues/2743#issuecomment-1018280042

github-actions[bot] commented 2 years ago

Stale issue message

jsmeix commented 2 years ago

With current master code and in local.conf

LIBS+=( /usr/lib*/qt5/* )

I get (excerpts)

# usr/sbin/rear -D mkrescue
...
/bin/xmlpatternsvalidator-qt5 requires additional libraries (fatal error)
/bin/xmlpatterns-qt5 requires additional libraries (fatal error)
/bin/sdpscanner requires additional libraries (fatal error)
/usr/lib64/qt5/libexec/QtWebNetworkProcess requires additional libraries
/usr/lib64/qt5/libexec/QtWebEngineProcess requires additional libraries
/usr/lib64/qt5/libexec/QtWebStorageProcess requires additional libraries
/usr/lib64/qt5/libexec/QtWebPluginProcess requires additional libraries
/usr/lib64/qt5/libexec/QtWebProcess requires additional libraries
/usr/lib64/qt5/plugins/styles/adwaita.so requires additional libraries
/usr/lib64/qt5/plugins/styles/libqgtk2style.so requires additional libraries
...
/usr/lib64/qt5/bin/qwebengine_convert_dict requires additional libraries (fatal error)
...
/usr/lib64/qt5/qml/Qt/labs/settings/libqmlsettingsplugin.so requires additional libraries
/usr/lib64/qt5/qml/Qt/labs/location/liblocationlabsplugin.so requires additional libraries
/usr/lib64/qt5/qml/Qt/labs/wavefrontmesh/libqmlwavefrontmeshplugin.so requires additional libraries
/usr/lib64/qt5/qml/Qt/labs/qmlmodels/liblabsmodelsplugin.so requires additional libraries
/usr/lib64/qt5/qml/Qt/labs/folderlistmodel/libqmlfolderlistmodelplugin.so requires additional libraries
/usr/lib64/qt5/qml/Qt/labs/sharedimage/libsharedimageplugin.so requires additional libraries
/usr/lib64/qt5/qml/QtNfc/libdeclarative_nfc.so requires additional libraries
/usr/lib64/qt5/qml/QtWebChannel/libdeclarative_webchannel.so requires additional libraries
/usr/lib64/qt5/qml/QtLocation/libdeclarative_location.so requires additional libraries
/usr/lib64/qt5/qml/QtQuick.2/libqtquick2plugin.so requires additional libraries
/usr/lib64/qt5/qml/QtQml/StateMachine/libqtqmlstatemachine.so requires additional libraries
/usr/lib64/qt5/qml/QtQml/Models.2/libmodelsplugin.so requires additional libraries
...
ReaR recovery system in '/var/tmp/rear.Gv9A0hpm1JnNZ5n/rootfs' needs additional libraries, check /root/rear.github.master/var/log/rear/rear-linux-h9wr.log for details
Build area kept for investigation in /var/tmp/rear.Gv9A0hpm1JnNZ5n, remove it when not needed
Testing that the existing programs in the PROGS array can be found as executable command within the recovery system
Testing that each program in the REQUIRED_PROGS array can be found as executable command within the recovery system
ERROR: ReaR recovery system in '/var/tmp/rear.Gv9A0hpm1JnNZ5n/rootfs' not usable (required libraries are missing)

build/default/990_verify_rootfs.sh shows 'requires additional libraries' in particular for all the 104 *.so files below /usr/lib64/qt5/ that exist on my openSUSE Leap 15.3 system.

The programs in the recovery system

/bin/xmlpatternsvalidator-qt5
/bin/xmlpatterns-qt5
/bin/sdpscanner

are there because they are on the original system as

# ls -l /usr/lib64/qt5/bin/xmlpatternsvalidator-qt5 /usr/lib64/qt5/bin/xmlpatterns-qt5 /usr/lib64/qt5/bin/sdpscanner

lrwxrwxrwx ... /usr/lib64/qt5/bin/sdpscanner -> ../../../bin/sdpscanner
lrwxrwxrwx ... /usr/lib64/qt5/bin/xmlpatterns-qt5 -> ../../../bin/xmlpatterns-qt5
lrwxrwxrwx ... /usr/lib64/qt5/bin/xmlpatternsvalidator-qt5 -> ../../../bin/xmlpatternsvalidator-qt5

This partially contradicts what I wrote above in https://github.com/rear/rear/issues/2743#issuecomment-1022133392 because when build/default/990_verify_rootfs.sh shows 'requires additional libraries' for all the 104 *.so files below /usr/lib64/qt5/ that exist on my openSUSE Leap 15.3 system it means all the 104 *.so files below /usr/lib64/qt5/ were copied into the recovery system in build/GNU/Linux/390_copy_binaries_libraries.sh

What did not happen in 390_copy_binaries_libraries.sh was that other libraries that are required by the 104 *.so files below /usr/lib64/qt5/ were also automatically copied into the recovery system.

As far as I see this is because the RequiredSharedObjects function is not run for each one of those 104 *.so files but for to what the bash globbing pattern /usr/lib*/qt5/* evaluates because the log file contains

+ source /root/rear.github.master/usr/share/rear/build/GNU/Linux/390_copy_binaries_libraries.sh
...
++ all_libs=("${LIBS[@]}" $( RequiredSharedObjects "${all_binaries[@]}" "${LIBS[@]}" ))
+++ RequiredSharedObjects /bin/true ... /usr/lib64/qt5/bin /usr/lib64/qt5/libexec /usr/lib64/qt5/plugins /usr/lib64/qt5/qml ...
jsmeix commented 2 years ago

With

LIBS+=( /usr/lib*/qt5/* )
PROGS+=( xmlpatternsvalidator-qt5 xmlpatterns-qt5 sdpscanner /usr/lib64/qt5/bin/qwebengine_convert_dict )

it does no longer error out and I get (excerpts)

# usr/sbin/rear -D mkrescue
...
/usr/lib64/qt5/libexec/QtWebNetworkProcess requires additional libraries
/usr/lib64/qt5/libexec/QtWebEngineProcess requires additional libraries
...
/usr/lib64/qt5/qml/QtQml/StateMachine/libqtqmlstatemachine.so requires additional libraries
/usr/lib64/qt5/qml/QtQml/Models.2/libmodelsplugin.so requires additional libraries
ReaR recovery system in '/var/tmp/rear.7cERtfkwrRNfN2p/rootfs' needs additional libraries, check /root/rear.github.master/var/log/rear/rear-linux-h9wr.log for details
Testing that the existing programs in the PROGS array can be found as executable command within the recovery system
Testing that each program in the REQUIRED_PROGS array can be found as executable command within the recovery system
Running 'pack' stage ======================
jsmeix commented 2 years ago

With only

PROGS+=( xmlpatternsvalidator-qt5 xmlpatterns-qt5 sdpscanner /usr/lib64/qt5/bin/qwebengine_convert_dict )

i.e. no longer LIBS+=( /usr/lib*/qt5/* ) all looks well for "rear -D mkrescue" i.e. no longer any "requires additional libraries" message.

Those programs appear in the recovery system as

/var/tmp/rear.Gvnjaj2VsXKe8nu/rootfs/bin/xmlpatternsvalidator-qt5
/var/tmp/rear.Gvnjaj2VsXKe8nu/rootfs/bin/xmlpatterns-qt5
/var/tmp/rear.Gvnjaj2VsXKe8nu/rootfs/bin/sdpscanner
/var/tmp/rear.Gvnjaj2VsXKe8nu/rootfs/bin/qwebengine_convert_dict

so all looks well.

jsmeix commented 2 years ago

As far as I currently see my above tests https://github.com/rear/rear/issues/2743#issuecomment-1128628990 and https://github.com/rear/rear/issues/2743#issuecomment-1128657211 and https://github.com/rear/rear/issues/2743#issuecomment-1128660068 prove what I had written above in https://github.com/rear/rear/issues/2743#issuecomment-1018280042

Plus my above conclusion therein in different words from another point of view ("including programs" point of view):

Needed programs should normally ony specified via

PROGS+=( non_mandatory_program )
REQUIRED_PROGS+=( mandatory_program )

But normally there is no need to specify LIBS for them because needed libraries for "normally" linked programs get automatically included. Only libraries that are loaded by executable programs via things like dlopen needed to be specified in LIBS.

jsmeix commented 2 years ago

With only

LIBS+=( /usr/lib64/qt5/*/*/*.so /usr/lib64/qt5/*/*/*/*.so /usr/lib64/qt5/*/*/*/*/*.so )

all looks well for "rear -D mkrescue" i.e. no longer any "requires additional libraries" message.

All the 104 *.so files below /usr/lib64/qt5/ were copied into the recovery system plus all their required other libraries.

Via https://github.com/rear/rear/commit/f8f6c1b94a20a0dfa3b81aa0d23a9abcffa5b356 I explained in default.conf how to use LIBS properly.

github-actions[bot] commented 2 years ago

Stale issue message

github-actions[bot] commented 2 years ago

Stale issue message

github-actions[bot] commented 1 year ago

Stale issue message

github-actions[bot] commented 1 year ago

Stale issue message

github-actions[bot] commented 1 year ago

Stale issue message

github-actions[bot] commented 1 year ago

Stale issue message

github-actions[bot] commented 1 year ago

Stale issue message

github-actions[bot] commented 1 year ago

Stale issue message

github-actions[bot] commented 10 months ago

Stale issue message

github-actions[bot] commented 8 months ago

Stale issue message

github-actions[bot] commented 6 months ago

Stale issue message

github-actions[bot] commented 4 months ago

Stale issue message

jsmeix commented 3 months ago

Nothing urgent here, so postponed to 'ReaR v3.1' milestone...

github-actions[bot] commented 1 month ago

Stale issue message