Closed nmaheve closed 2 years ago
@nmaheve Please fill in a correct issue title and something with the description when you fill in a template, otherwise, you make it hard for us to find out what the problem is. Remember, we do this for free! So this is my best guess:
2021-08-10 13:49:44.409619164 grub2-mkstandalone may fail to make a bootable EFI image of GRUB2 (no /usr/*/grub*/x86_64-efi/moddep.lst file)
2021-08-10 13:49:44.413801903 GRUB2 modules to load: fat part_gpt xfs
2021-08-10 13:49:44.434441145 Some messages from /var/tmp/rear.EfGoBXYia75mbzQ/tmp/rear.mkrescue.stdout_stderr since the last called script 250_populate_efibootimg.sh:
/usr/share/rear/lib/_input-output-functions.sh: line 510: type: grub-mkstandalone: not found
/usr/share/rear/lib/_input-output-functions.sh: line 510: type: grub-probe: not found
grub2-mkstandalone: error: /usr/lib/grub/x86_64-efi/modinfo.sh doesn't exist. Please specify --target or --directory.
2021-08-10 13:49:44.439265888 ERROR: Failed to make bootable EFI image of GRUB2 (error during grub2-mkstandalone of /var/tmp/rear.EfGoBXYia75mbzQ/tmp/mnt/EFI/BOOT/BOOTX64.efi)
Can you confirm package grub2-efi-x64-modules is present on your system? See also issue #2600
This is not installed on server
This is not installed on server
@nmaheve install the package and retry, ok?
Classification: Confidential
Sure will install and let u know
Thanks , Nikhil Maheve
we have insatlled
[root@testl07dc ~]# rpm -q grub2-efi-x64-modules grub2-efi-x64-modules-2.02-0.87.el7_9.6.noarch
We have installed the package on server ..
Here is the output.. @. ~]# rpm -qa | grep grub2-efi-x64-modules grub2-efi-x64-modules-2.02-0.87.el7_9.6.noarch @. ~]#
please refer the log file after installing package i have run the command log.txt
Classification: Confidential Hello Support team
Do we have any update on this .. We have installed the package on server , and after installation I have tried to run the rear -D mkrescue but still we are same error .
I have uploaded the log file as well on portal , requesting you to please help us here to resolve this issue
Thanks , Nikhil Maheve
@nmaheve The Grub2 creation was successful, however, you have some missing libraries:
Testing each binary with 'ldd' and look for 'not found' libraries within the recovery system
/usr/lib64/eog/plugins/libfullscreen.so requires additional libraries
libeog.so => not found
/usr/lib64/eog/plugins/libreload.so requires additional libraries
libeog.so => not found
/usr/lib64/eog/plugins/libstatusbar-date.so requires additional libraries
libeog.so => not found
/usr/lib64/samba/pdb/smbpasswd.so requires additional libraries
Try to change the following entry in the /etc/rear/.local.conf file:
COPY_AS_IS_CDM=( /etc/rubrik /usr/bin/rubrik /var/log/rubrik /etc/pki /usr/lib64/*)
log.txt please have done the changes we have copied complete COPY_AS_IS_CDM=( /etc/rubrik /usr/bin/rubrik /var/log/rubrik /etc/pki /usr/lib64/*) in .local.conf file
please suggest
hello Support ,
please suggest we are waiting for your response ..
@nmaheve Please check where on your live system these missing libraries, e.g. libeog.so, resides. E.g. try
find / -name "libeog.so*"
to find the path. If that path is different then /usr/lib64
add it to the variable COPY_AS_IS_CDM
(in assumption you are still using BACKUP=CDM
)
here is the output of local.conf file
OUTPUT=ISO
# # Specifies CDM as the backup and recovery application
#BACKUP=CDM
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib64/bind9-export"
COPY_AS_IS_CDM=( /etc/rubrik /usr/bin/rubrik /var/log/rubrik /etc/pki /usr/lib64/*
we have commented the backup=CDM after that we have run the rear -D mkrescue and the output is attached here log.txt
@nmaheve If you comment out BACKUP=CDM
then the line COPY_AS_IS_CDM
will not be used. And, did you forget to end it with )
?
Within ReaR everything is a bash script, so you can test it with bash -n /etc/rear/local.conf
(should not complain).
What is the content of ls /etc/ld.so.conf.d/
? I think within ReaR root fs the /etc/ld.so.conf file is missing a library path. For example, for Legato we had to add something like /usr/share/rear/build/NSR/default/550_add_nsr_libs_to_ldsoconf.sh
to add some missing library paths. Could be an equivalent issue.
Concerning the original issue.
Can you confirm package grub2-efi-x64-modules is present on your system? See also issue #2600
There are several reports of the same issue, like #1193 and #2524 . As you can see from https://github.com/rear/rear/issues/1193#issuecomment-451450705, I proposed to add a warning that hints what package to install, but at the end a more generic message was chosen: https://github.com/rear/rear/issues/1193#issuecomment-452617591 that does not mention the specific package:
If we detect something is missing, we will not tell the user what software package (e.g. a rpm or deb package or whatever else) he must install to get the missing piece because we cannot continuously track and maintain for all those various kind of different Linux distributions what particular software package provides what program or file in what version of each Linux distribution.
@pcahyna to avoid misunderstandings:
My statement in https://github.com/rear/rear/issues/1193#issuecomment-452617591 that we will not tell the user what software package is missing was meant for the generic code in ReaR, cf. what @gdha wrote in https://github.com/rear/rear/issues/1193#issuecomment-451450705 "if we decide to add this we should verify the OS version".
So we could have distribution specific code provided:
Distribution specific code is properly conditioned.
See the SetOSVendorAndVersion
function in lib/config-functions.sh
what variables are set how to which vaules to distinguish between
different distributions and distribution versions.
See layout/prepare/Linux-s390/205_s390_enable_disk.sh
for an example of distribution specific code inside a script.
AND
There is someone who continuously tracks and maintains distribution specific code to keep it up to date in the future.
By the way as an additional side note in general regarding the word "WARNING" in messages see https://schlomo.schapiro.org/2015/04/warning-is-waste-of-my-time.html and for background information see https://github.com/rear/rear/issues/564#issuecomment-86188528 and the subsequent comments therein.
@jsmeix thanks for the clarification, this separation of distribution-specific code makes sense. I will resubmit the change in a way that respects the separation. Concerning WARNING, I think it can be justified if it explains an error that is expected to happen in the next step (so that in the output you see the error immediately preceded by the warning that explains its likely cause and action to take). Of course, the explanatory message could be printed only when the actual error occurs, but then it will be harder to maintain it separately in a distribution-specific code.
Thank you I already tried that method to exclude those library and getting error "rubrik cluster couldn't connect to the agent while run restore using that ISO (size is decreased to 733MB to 300MB)
Hello @DamaniN, you wrote the Rubrik CDM support, could you please help here?
@bbabaria,
Please refer to https://github.com/rear/rear/issues/2685#issuecomment-935098986 to troubleshoot the issue you are seeing.
@nmaheve,
Please update your LD_LIBRARY_PATH
in the local.conf
file to include:
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib64/bind9-export:/usr/lib64/eog:/usr/lib64/python3.6/site-packages:/usr/lib64/samba:/usr/lib64/firefox:/usr/lib64/python3.6/site-packages/haw key:/usr/lib64/mutter-4"
In general FYI regarding LD_LIBRARY_PATH:
In general using LD_LIBRARY_PATH is known to cause troubles, cf. https://github.com/rear/rear/issues/1907#issuecomment-434215448 which links to https://www.hpc.dtu.dk/?page_id=1180 "LD_LIBRARY_PATH – or: How to get yourself into trouble!" which further links to http://xahlee.info/UnixResource_dir/_/ldpath.html "Why LD_LIBRARY_PATH is bad"
In particular using export LD_LIBRARY_PATH="..."
is usually really bad, see
https://www.hpc.dtu.dk/?page_id=1180
that reads (excerpt)
Do NOT do:
$ export LD_LIBRARY_PATH=/path/to/lib1:/path/to/lib2:/path/to/lib3
$ ./myprog
since this will pollute the shell environment for all consecutive commands!
Never put LD_LIBRARY_PATH in your login profiles!
In that way you will expose all the applications you start
to this – probably problematic – path!
See also https://github.com/rear/rear/issues/1927 therein in particular starting at https://github.com/rear/rear/issues/1927#issuecomment-430254499
Hi Pcahyna,
Please find the attached log files. Thank you, rear-njidlpuppet01.log
We were not able to create ISO of the server. While running the command, getting below error on RHEL 7.9.
ot@testl07dc ~]# rear -d -v mkbackup Relax-and-Recover 2.4 / Git Using log file: /var/log/rear/rear-testl07dc.log ERROR: EFI_STUB="" is incorrect option value pair. Aborting due to an error, check /var/log/rear/rear-testl07dc.log for details Exiting rear mkbackup (PID 25024) and its descendant processes Running exit tasks Terminated
@nmaheve can you post an update on how you were able to create the ISO on the original server?
I spent some time with the team this afternoon after they were able to create an ISO from the RHEL 7.9 server. Attempting to recover presented some additional challenges though.
Appended to the default /etc/rear/local.conf was:
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib64/bind9-export:/usr/lib64/eog:/usr/lib64/python3.6/site-packages:/usr/lib64/samba:/usr/lib64/firefox:/usr/lib64/python3.6/site-packages/hawkey:/usr/lib64/mutter-4"
After the recovery attempt the logs showed the following:
Classification: Confidential Hello Support ,
We were facing issue while creating the As per Redhat recommendation, we uninstall the rear packages provided by Rubrik and install the required packages from Redhat repository. Then , tried to create backup using CDM backup option and run command rear -v mkresuce to create ISO on path /var/lib/rear/output. It ran successfully without any errors and ISO image has been created as backup on the server.
Please refer the attached case history
Thanks , Nikhil Maheve
From: marcus9467 @.> Sent: Friday, November 5, 2021 11:03 PM To: rear/rear @.> Cc: Nikhil Kumar Maheve @.>; Mention @.> Subject: Re: [rear/rear] Failed to make bootable EFI image of GRUB2 (#2674)
[CAUTION: This Email is from outside the Organization. Unless you trust the sender, Don't click links or open attachments as it may be a Phishing email, which can steal your Information and compromise your Computer.]
@nmahevehttps://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnmaheve&data=04%7C01%7Cnikhilkumar.maheve%40hcl.com%7C8600e991c28e4afc0efe08d9a0825a56%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C637717303997437577%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JajkbjrqxxY%2FBSpDRQZfwDoQzvwkSfssBZzGA8SnRY8%3D&reserved=0 can you post an update on how you were able to create the ISO on the original server?
I spent some time with the team this afternoon after they were able to create an ISO from the RHEL 7.9 server. Attempting to recover presented some additional challenges though.
Appended to the default /etc/rear/local.conf was: export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib64/bind9-export:/usr/lib64/eog:/usr/lib64/python3.6/site-packages:/usr/lib64/samba:/usr/lib64/firefox:/usr/lib64/python3.6/site-packages/hawkey:/usr/lib64/mutter-4"
After the recovery attempt the logs showed the following: [image (1)]https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F41637724%2F140553572-3f050bae-403f-44b8-8ca0-a27987230cf2.png&data=04%7C01%7Cnikhilkumar.maheve%40hcl.com%7C8600e991c28e4afc0efe08d9a0825a56%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C637717303997447536%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=xvJsmNT65YoblVCmJkGpw%2B7IWMWppk6PbxlQ7%2B4e0HA%3D&reserved=0 [image (2)]https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F41637724%2F140553756-686293f3-b34b-4f0d-b077-557767822944.png&data=04%7C01%7Cnikhilkumar.maheve%40hcl.com%7C8600e991c28e4afc0efe08d9a0825a56%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C637717303997447536%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1I2cXTwm5PXMuo6ZBQuOIobEWrB0Z%2BMCvieL%2Fq5FBJ0%3D&reserved=0
- You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frear%2Frear%2Fissues%2F2674%23issuecomment-962084724&data=04%7C01%7Cnikhilkumar.maheve%40hcl.com%7C8600e991c28e4afc0efe08d9a0825a56%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C637717303997457498%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3Mkr6%2B4uQd92VHK0fZ6qtZayhRGNmW1vJne%2BInMzxtI%3D&reserved=0, or unsubscribehttps://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAVLQTPNHRWBMPMWHLLXFZG3UKQPOVANCNFSM5C323CSA&data=04%7C01%7Cnikhilkumar.maheve%40hcl.com%7C8600e991c28e4afc0efe08d9a0825a56%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C637717303997457498%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=TyHWUwMnKOATK9%2BNLaAy9Wz%2Fwj3bY9n1%2FJKckzeq%2F0I%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Cnikhilkumar.maheve%40hcl.com%7C8600e991c28e4afc0efe08d9a0825a56%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C637717303997467451%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=mxhR8mb%2FdvD8qFN81QEfnnOfjVlPUsE1ooIjVa1aoGA%3D&reserved=0 or Androidhttps://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Cnikhilkumar.maheve%40hcl.com%7C8600e991c28e4afc0efe08d9a0825a56%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C637717303997467451%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=YzlEy1ApU9ku8iCuJMbEWz5aMAMpZSNo0JOrvuQSc8A%3D&reserved=0.
::DISCLAIMER::
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
Hello Team,
Thanks for the sosreport,
$ cat df Filesystem 1K-blocks Used Available Use% Mounted on /dev/mapper/rhel-root 104806400 3600440 101205960 4% / devtmpfs 16377592 0 16377592 0% /dev tmpfs 16405900 0 16405900 0% /dev/shm tmpfs 16405900 34100 16371800 1% /run /dev/mapper/rhel-var 20961280 560368 20400912 3% /var /dev/mapper/rhel-tmp 10475520 144696 10330824 2% /tmp /dev/sda2 2037760 165284 1872476 9% /boot /dev/sda1 1021984 9964 1012020 1% /boot/efi /dev/mapper/rhel-home 20961280 143436 20817844 1% /home /dev/mapper/rhel-opt 20961280 33328 20927952 1% /opt
I am seeing data on the system,
The system is not booting apparently, let' have you do the following:
If these exist and there are boot file, let's have you re-install grub.
How to reinstall GRUB and GRUB2 on UEFI-based machines? - https://access.redhat.com/solutions/3486741
Then try to re-boot.
RHEL 8 | Any Cloud. Any Workload. One OS. https://www.redhat.com/en/enterprise-linux-8 https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000cjcvnQAA Bhargava, Nishant Wed, Sep 22, 2021, 10:58:50 PM GMT+5:30 Attached file to this case. SHA-256 f3f19388babf990d8e3e79f7b695484f26f86ccdc215df0eb8e41ff69a13c6ac
Description Enter a description https://access.redhat.com/support/cases/#/case/03025370/discussion?attachmentId=a092K00002l0zOmQAI Bhargava, Nishant Wed, Sep 22, 2021, 10:58:44 PM GMT+5:30 Hi,
Please find the manual sosreport. https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000cjP8HQAU Red Hat associate comment Kielian, Meredith Wed, Sep 22, 2021, 10:20:12 PM GMT+5:30 Hello Team,
Let's have you try to run a manual sosreport:
Note: You may want to bring up network and then copy script from another system.
Sosreport fails. What data should I provide in its place?
https://access.redhat.com/solutions/68996
RHEL 8 | Any Cloud. Any Workload. One OS. https://www.redhat.com/en/enterprise-linux-8 https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000cjOctQAE Bhargava, Nishant Wed, Sep 22, 2021, 12:02:31 AM GMT+5:30 Hi,
I tried to collect sosreport by running the command but getting an error while running the "sosreport" command. https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000cjASxQAM Bhargava, Nishant Wed, Sep 22, 2021, 12:00:43 AM GMT+5:30 Attached file to this case. SHA-256 a13c12ebb209b12a2fb7e62d870482ee0e68415bdaa1f1840c6109bdaec15b5b
Description Enter a description https://access.redhat.com/support/cases/#/case/03025370/discussion?attachmentId=a092K00002l0v8gQAA Red Hat associate comment Kielian, Meredith Tue, Sep 21, 2021, 10:37:40 PM GMT+5:30 Hello Team,
the server got crashed and can't fetch any log from the server.
Please suggest an alternate solution as I have already sent the screenshot of the issue.
So is the error in the screenshot from the restored system or from when you boot from the ReaR ISO?
Also if the error is from the restored syste, you can try to rescue mode to retrieve the log files and then enable networkign to pull those files off.
RESCUE MODE:
How to boot Red Hat Enterprise Linux to Rescue Mode for Data Collection (sosreport, vmcore, etc.) - https://access.redhat.com/articles/3405661
ENABLE NETWORK IN RESCUE MODE:
You may need to activate networking in rescue mode in order to move the sosreport to a system than can uploade to red Hat, I have provided instructions below:
Enabling networking in rescue environment without chrooting -
https://access.redhat.com/solutions/2626631
Please feel free to update the case with any comments, concerns, or feedback on the above.
RHEL 8 | Any Cloud. Any Workload. One OS. https://www.redhat.com/en/enterprise-linux-8 https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000cj9WKQAY Bhargava, Nishant Tue, Sep 21, 2021, 08:25:59 PM GMT+5:30 Hi, I haven't tried with non-CMD. the server got crashed and can't fetch any log from the server.
Please suggest an alternate solution as I have already sent the screenshot of the issue. https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000cj7WxQAI Red Hat associate comment Kielian, Meredith Tue, Sep 21, 2021, 04:17:09 AM GMT+5:30 Hello Team,
It looks like it is not even booting up to the ReaR recovery menu.
Have you tried booting the ISO made that is non-CDM?
It should boot to a menu at least asking for restore options.
Maybe attach the rear logs made when you made the CDM backup and I can check the log files when it created the ISO to see if there are any warnings/errors, thanks.
Please let me know, thanks.
RHEL 8 | Any Cloud. Any Workload. One OS. https://www.redhat.com/en/enterprise-linux-8 https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000cixOMQAY Bhargava, Nishant Tue, Sep 21, 2021, 12:58:24 AM GMT+5:30 Hi,
The server got crashed while recovery and recovery was also not successful. I have attached the screenshot of the current status where the server got stuck during the recovery process. https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000civjWQAQ Bhargava, Nishant Tue, Sep 21, 2021, 12:57:30 AM GMT+5:30 Attached file to this case. SHA-256 4124961500781a109be08d794654af3f09100a76fdaa3c3f0dc1b5fdf9b26522
Description Enter a description https://access.redhat.com/support/cases/#/case/03025370/discussion?attachmentId=a092K00002l0qixQAA Red Hat associate comment Kielian, Meredith Fri, Sep 17, 2021, 03:07:53 AM GMT+5:30 Hello Team,
Glad to hear it, now go ahead and try the backup using the CDM backup option.
If that works then you should be able to continue forward.
RHEL 8 | Any Cloud. Any Workload. One OS. https://www.redhat.com/en/enterprise-linux-8 https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000ciQRGQA2 Bhargava, Nishant Fri, Sep 17, 2021, 02:32:10 AM GMT+5:30 Hello Meredith ,
It seems this ran fine this time. OUTPUT=ISO OUTPUT_URL=nfs://10.137.13.235/storage BACKUP=NETFS BACKUP_URL=nfs://10.137.13.235/storage BACKUP_PROG_EXCLUDE=("${BACKUP_PROG_EXCLUDE[@]}" '/media' '/var/tmp' '/var/crash') @. tmp]# vi /etc/rear/local.conf @. tmp]# rear -d -v mkbackup Relax-and-Recover 2.4 / Git Using log file: /var/log/rear/rear-testl07dc.log Using backup archive '/tmp/rear.zvERLpPqfgYduUF/outputfs/testl07dc/backup.tar.gz' Using UEFI Boot Loader for Linux (USING_UEFI_BOOTLOADER=1) Creating disk layout Using guessed bootloader 'EFI' (found in first bytes on /dev/sda) Creating root filesystem layout Handling network interface 'eno1' eno1 is a physical device Handled network interface 'eno1' Skipping 'virbr0': not bound to any physical interface. To log into the recovery system via ssh set up /root/.ssh/authorized_keys or specify SSH_ROOT_PASSWORD Trying to find what to use as UEFI bootloader... Trying to find a 'well known file' to be used as UEFI bootloader... Using '/boot/efi/EFI/redhat/grubx64.efi' as UEFI bootloader file Copying logfile /var/log/rear/rear-testl07dc.log into initramfs as '/tmp/rear-testl07dc-partial-2021-09-16T16:40:39-0400.log' Copying files and directories Copying binaries and libraries Copying kernel modules Copying all files in /lib*/firmware/ Creating recovery/rescue system initramfs/initrd initrd.cgz with gzip default compression Created initrd.cgz with gzip default compression (281689798 bytes) in 26 seconds Making ISO image Wrote ISO image: /var/lib/rear/output/rear-testl07dc.iso (313M) Copying resulting files to nfs location Saving /var/log/rear/rear-testl07dc.log as rear-testl07dc.log to nfs location Creating tar archive '/tmp/rear.zvERLpPqfgYduUF/outputfs/testl07dc/backup.tar.gz' Archived 2013 MiB [avg 8964 KiB/sec] OK Archived 2013 MiB in 231 seconds [avg 8925 KiB/sec] Exiting rear mkbackup (PID 14863) and its descendant processes Running exit tasks
Should I try to recover from backup.
Regards, Nishant Bhargava https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000ciQAyQAM Red Hat associate comment Kielian, Meredith Thu, Sep 16, 2021, 11:55:24 PM GMT+5:30 Hello Team,
From everything I am seeing in the logs and the packages, I am thinking there is a conflict with how ReaR was originally installed.
Since it was not installed from a Red Hat repo initially we may have a file that was not updated when the Red Hat version was installed.
We may need to remove ReaR and then manually re-move all the remaining components.
I am thinking the previous ReaR install has an extra script that is running, specifically the script: [ /usr/share/rear/init/default/010_EFISTUB_check.sh ]
Since when I check the rear package:
rear-2.4-13.el7.x86_64
[ NO RESULTS ]
So the rear is seeing this script and trying to run it, BUT, the version of rear is not expecting that file to exist.
So let's remove rear and then clear out all the rear files, then re-install it:
Remove Rear
Clear out all extra rear files:
Make a backup of them first:
Remove all rear files:
You can either manually clear out the following files/directories; /etc/rear /usr/sbin/rear /usr/share/rear /var/lib/rear
Or run the following commands: IMPORTANT NOTE: Be VERY VERY careful as using "rm -rf" with the wrong options can remove files you didn't intend to
$ yum install rear
Thanks.
RHEL 8 | Any Cloud. Any Workload. One OS. https://www.redhat.com/en/enterprise-linux-8 https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000ciOK8QAM Bhargava, Nishant Thu, Sep 16, 2021, 01:09:19 AM GMT+5:30 Hello Meredith ,
PFA the logs file further for verbose option.
Regards, Nishant Bhargava https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000ci9pjQAA Bhargava, Nishant Thu, Sep 16, 2021, 01:09:15 AM GMT+5:30 Attached file to this case. SHA-256 a7c58bfc228749928c31b9970e11c74013d096143406e8b5d7a0ede7dff77ae5
Description Enter a description https://access.redhat.com/support/cases/#/case/03025370/discussion?attachmentId=a092K00002l0Z8qQAE Red Hat associate comment Kielian, Meredith Wed, Sep 15, 2021, 11:57:41 PM GMT+5:30 Hello Team,
Let's have you run the following ( with the CDM still excluded and a "baseline ReaR config )
The "-D" option should provide more debugging info in the log file, thanks!
RHEL 8 | Any Cloud. Any Workload. One OS. https://www.redhat.com/en/enterprise-linux-8 https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000ci97KQAQ Bhargava, Nishant Wed, Sep 15, 2021, 08:43:39 PM GMT+5:30 Attached file to this case. SHA-256 999cdc3bae5df6a6646b1a8ea8326efcbabce2660b46420cc50e2328e8e858da
Description Enter a description https://access.redhat.com/support/cases/#/case/03025370/discussion?attachmentId=a092K00002l0YAfQAM Bhargava, Nishant Wed, Sep 15, 2021, 08:42:39 PM GMT+5:30 Hello Meredith ,
We are getting the same errors even after excluding CDM.
@.*** ~]# rear -d -v mkbackup Relax-and-Recover 2.4 / Git Using log file: /var/log/rear/rear-testl07dc.log ERROR: EFI_STUB="" is incorrect option value pair. Aborting due to an error, check /var/log/rear/rear-testl07dc.log for details Exiting rear mkbackup (PID 25024) and its descendant processes Running exit tasks Terminated
##############################################################################
@.*** ~]# cat /etc/rear/local.conf
#
OUTPUT=ISO OUTPUT_URL=nfs://10.137.13.235/storage BACKUP=NETFS BACKUP_URL=nfs://10.137.13.235/storage BACKUP_PROG_EXCLUDE=("${BACKUP_PROG_EXCLUDE[@]}" '/media' '/var/tmp' '/var/crash')
Attaching the logs file as well.
Regards, Nishant Bhargava https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000ci6SXQAY Red Hat associate comment Kielian, Meredith Wed, Sep 15, 2021, 04:29:32 AM GMT+5:30 Hello Team,
Here is the results from the package check:
avahi.x86_64 glusterfs.x86_64 libreswan.x86_64 pulseaudio.x86_64
I am not seeing much related to rear.
I would like to take "CDM" out of the equation.
Can you try a test backup using the example provided in this document:
What is Relax and Recover(ReaR) and how can I use it for disaster recovery? -
https://access.redhat.com/solutions/2115051
I want to see if we can do a regular backup without pointing to CDM.
Thanks.
RHEL 8 | Any Cloud. Any Workload. One OS. https://www.redhat.com/en/enterprise-linux-8 https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000cEYyMQAW Bhargava, Nishant Tue, Sep 14, 2021, 07:40:30 PM GMT+5:30 Hello Meredith ,
PFA the required file.
Regards, Nishant Bhargava https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000cET7bQAG Bhargava, Nishant Tue, Sep 14, 2021, 07:39:41 PM GMT+5:30 Attached file to this case. SHA-256 d19cb89c5cafa302c42235139f37d21d94474f4d19bfb18d8eec1c31a562ec97
Description Enter a description https://access.redhat.com/support/cases/#/case/03025370/discussion?attachmentId=a092K00002l0RlvQAE Red Hat associate comment Kielian, Meredith Tue, Sep 14, 2021, 12:46:51 AM GMT+5:30 Hello Team,
Let's take a look at the local.conf
$ cat ../../../etc/rear/local.conf
#
OUTPUT=ISO
BACKUP=CDM
Looks like we may be missing some options.
Checking error:
$ cat rear-testl07dc.log
2021-09-13 13:39:59.875131702 Relax-and-Recover 2.4 / Git
2021-09-13 13:39:59.876475642 Command line options: /usr/sbin/rear -v mkrescue
2021-09-13 13:39:59.877618021 Using log file: /var/log/rear/rear-testl07dc.log
2021-09-13 13:39:59.879017647 Including /etc/rear/os.conf
2021-09-13 13:39:59.882579479 Including conf/Linux-i386.conf
2021-09-13 13:39:59.884174965 Including conf/GNU/Linux.conf
2021-09-13 13:39:59.898543021 Including /etc/rear/local.conf
2021-09-13 13:39:59.901269570 ======================
2021-09-13 13:39:59.902644761 Running 'init' stage
2021-09-13 13:39:59.903840161 ======================
2021-09-13 13:39:59.912079690 Including init/default/005_verify_os_conf.sh
2021-09-13 13:39:59.916715260 Including init/default/010_EFISTUB_check.sh
2021-09-13 13:39:59.918078326 ERROR: EFI_STUB="" is incorrect option value pair.
==== Stack trace ====
Trace 0: /usr/sbin/rear:502 main
Trace 1: /usr/share/rear/lib/framework-functions.sh:101 SourceStage
Trace 2: /usr/share/rear/lib/framework-functions.sh:49 Source
Trace 3: /usr/share/rear/init/default/010_EFISTUB_check.sh:6 source
Message: EFI_STUB="" is incorrect option value pair.
== End stack trace ==
2021-09-13 13:39:59.924200758 Exiting rear mkrescue (PID 32311) and its descendant processes
2021-09-13 13:40:00.949189777 rear,32311 /usr/sbin/rear -v mkrescue
-rear,32453 /usr/sbin/rear -v mkrescue
-pstree,32454 -Aplau 32311
/usr/share/rear/lib/_input-output-functions.sh: line 157: kill: (32457) - No such process
2021-09-13 13:40:00.977665583 Running exit tasks
I did find this doc:
External Link: https://github.com/rear/rear/blob/master/doc/user-guide/16-Rubrik-CDM.adoc
Which has information on how to troubleshoot on ReaR and Rubrik
Let's Also get an rpm -Va to verify the integrity of all the package files on the system. ( especially the ReaR files )
The following command will run an rpm -Va and output into /tmp.
Please upload/attach the generated file [ /tmp/rpm_va.out ] to the case for further analysis.
RHEL 7/8
Once you have generated the file please upload the file: [ /tmp/rpm_va.out ] to the case.
If you would like to analyze it yourself you can check it against this Legend from the rpm man page.
LEGEND: ( source: rpm -Va )
S file Size differs
M Mode differs (includes permissions and file type)
5 digest (formerly MD5 sum) differs
D Device major/minor number mismatch
L readLink(2) path mismatch
U User ownership differs
G Group ownership differs
T mTime differs
P caPabilities differ
If you want to check which packages associated with those files are flagged you can run either of the following commands:
To get the full package name: ( command ran on same system the rpm_va.out was generated )
To get the package name: ( command ran on a different system than the on rpm_va.out was generated )
If you think replacing/reinstalling the package will correct the issue, you can run the following commands:
RHEL 5/6/7 System:
If the issue is with package file permissions you can run the following commands or follow the guide below:
Sets user/group ownership of files in the given package.
Sets permissions of files in the given package.
IMPORTANT: You must run "--setuguids" BEFORE running "--setperms"
How do I restore permissions of all the rpm packages on system to default? - https://access.redhat.com/solutions/281923
Please let us know if you have any additional questions or concerns, and thank you for using Red Hat.
RHEL 8 | Any Cloud. Any Workload. One OS. https://www.redhat.com/en/enterprise-linux-8 https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000cEHqyQAG Bhargava, Nishant Mon, Sep 13, 2021, 11:37:04 PM GMT+5:30 Attached file to this case. SHA-256 49eece32d9462a0eb2db159e734254237cf07f40ca72365be7b95415046e0850
Description Enter a description https://access.redhat.com/support/cases/#/case/03025370/discussion?attachmentId=a092K00002l0O9fQAE Bhargava, Nishant Mon, Sep 13, 2021, 11:36:59 PM GMT+5:30 Hi Meredith,
Herewith attached sosreport. https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000cEH4NQAW Red Hat associate comment Kielian, Meredith Mon, Sep 13, 2021, 11:28:41 PM GMT+5:30 Hello,
Go ahead and generate a fresh sosreport and I can check the generated rear log files included within it, thanks.
Thanks.
RHEL 8 | Any Cloud. Any Workload. One OS. https://www.redhat.com/en/enterprise-linux-8 https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000cEGqmQAG Bhargava, Nishant Mon, Sep 13, 2021, 11:11:14 PM GMT+5:30 Hi,
I have installed "rear" by yum but now we are getting a different error.
@. ~]# rpm -qa | grep rear rear-2.4-13.el7.x86_64 @. ~]# rpm -qa | grep genisoimage genisoimage-1.1.11-25.el7.x86_64 @. ~]# rpm -qa | grep syslinux syslinux-4.05-15.el7.x86_64 @. ~]# rear -v mkrescue Relax-and-Recover 2.4 / Git Using log file: /var/log/rear/rear-testl07dc.log ERROR: EFI_STUB="" is incorrect option value pair. Aborting due to an error, check /var/log/rear/rear-testl07dc.log for details Exiting rear mkrescue (PID 32311) and its descendant processes Running exit tasks Terminated @. ~]# date Mon Sep 13 13:40:03 EDT 2021 @. ~]# https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000cEGmOQAW Red Hat associate comment Kielian, Meredith Fri, Sep 3, 2021, 03:42:46 AM GMT+5:30 Hello,
My name is Meredith Kielian,
I am a member of the Red Hat North American team and I will be assisting on this case.
Let's check to make sure rear packages are installed:
$ cat sh_-crpm--nodigest-qa--qfNAME-VERSION-RELEASE._ARCH_INSTALLTIME_dateawk-Fprintf-59s_s_n_1_2sort-V | grep -e rear -e genisoimage -e syslinux genisoimage-1.1.11-25.el7.x86_64 Wed Aug 18 13:39:25 2021 syslinux-4.05-15.el7.x86_64 Thu Aug 19 10:28:56 2021
Okay so it looks like you are missing the "rear" package.
You will need to install it:
Also here is addtional information on ReaR below:
The following information covers ReaR ( Relax and Recover ) a system backup and recovery solution packaged with certain Versions of RHEL:
What is Relax and Recover(ReaR) and how can I use it for disaster recovery? -
https://access.redhat.com/solutions/2115051
Guides for each specific OS Versions are Below:
RHEL6: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/deployment_guide/ch-relax-and-recover_rear RHEL7: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/ch-relax-and-recover_rear RHEL8: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/configuring_basic_system_settings/index#basics-system-rescue-backup_getting-started-with-system-administration
RHEL Based Alternatives to ReaR:
How to backup and restore a whole Red Hat Enterprise Linux system with the dump/restore commands? - https://access.redhat.com/solutions/17525
Additional ReaR Information/Guides/Manuals:
Also for here is a good external resource for ReaR: http://relax-and-recover.org/documentation/ Note: This link is an External Link to the Open Source Development site for the latest version of ReaR as such some features may differ from the RHEL provided version.
You can also find a good Web/HTML documentation Guide for ReaR after installing it on a system here:
HTML Doc: /usr/share/doc/rear-
Please let us know if you have any additional questions or concerns, and thank you for using Red Hat.
RHEL 8 | Any Cloud. Any Workload. One OS. https://www.redhat.com/en/enterprise-linux-8 https://access.redhat.com/support/cases/#/case/03025370/discussion?commentId=a0a2K00000cCaEvQAK Bhargava, Nishant Thu, Sep 2, 2021, 08:54:53 PM GMT+5:30 Attached file to this case. SHA-256 83edfc28e631d76e5358700917e9ba1d82211628b940defca801ef4c34371394
Description Enter a description https://access.redhat.com/support/cases/#/case/03025370/discussion?attachmentId=a092K00002kza1oQAA
@nmaheve, It looks like the issue you are having now, as shown by the screenshots posted by @marcus9467, is with reformatting the disks on the recovery system. This is a not something that CDM does, but is part of the core ReaR code. Since this is not a CDM related issue, the people on this issue thread may not be able to help. Please create a new issue on this repo with the screenshots and description that @marcus9467 posted so that others who deal with that part of the ReaR code can help.
@nmaheve can this issue be closed for you?
I think https://github.com/rear/rear/pull/2747 also fixes the CDM related parts in this issue here.
I am facing a similar error, however I am running oracle linux 8.7. I do have grub2 utils installed. I have updated my LD_LIBRARY_PATH as stated in the thread above but still have the same error. How do you want me to proceed with a new issue or ?
Relax-and-Recover (ReaR) Issue Template
Fill in the following items before submitting a new issue (quick response is not guaranteed with free support):
ReaR version ("/usr/sbin/rear -V"): Relax-and-Recover 2.6 / Git
OS version ("cat /etc/os-release" or "lsb_release -a" or "cat /etc/rear/os.conf"):
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7" REDHAT_BUGZILLA_PRODUCT_VERSION=7.9 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="7.9"
[root@testl07dc ~]# cat /etc/rear/local.conf
This file etc/rear/local.conf is intended for the user's
manual configuration of Relax-and-Recover (ReaR).
For configuration through packages and other automated means
we recommend a separated file named site.conf next to this file
and leave local.conf as is (ReaR upstream will never ship a site.conf).
The default OUTPUT=ISO creates the ReaR rescue medium as ISO image.
You need to specify your particular backup and restore method for your data
as the default BACKUP=REQUESTRESTORE does not really do that (see "man rear").
Configuration variables are documented in /usr/share/rear/conf/default.conf
and the examples in /usr/share/rear/conf/examples/ can be used as templates.
ReaR reads the configuration files via the bash builtin command 'source'
so bash syntax like VARIABLE="value" (no spaces at '=') is mandatory.
Because 'source' executes the content as bash scripts you can run commands
within your configuration files, in particular commands to set different
configuration values depending on certain conditions as you need like
CONDITION_COMMAND && VARIABLE="special_value" || VARIABLE="usual_value"
but that means CONDITION_COMMAND gets always executed when 'rear' is run
so ensure nothing can go wrong if you run commands in configuration files.
Sets output to an be an ISO file
OUTPUT=ISO
Specifies CDM as the backup and recovery application
BACKUP=CDM
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/lib64/bind9-export"
[root@testl07dc ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 279.4G 0 disk ├─sda1 8:1 0 1000M 0 part ├─sda2 8:2 0 2G 0 part ├─sda3 8:3 0 90G 0 part │ ├─rhel-home 253:0 0 20G 0 lvm │ ├─rhel-root 253:1 0 100G 0 lvm │ ├─rhel-var 253:2 0 20G 0 lvm │ ├─rhel-tmp 253:3 0 10G 0 lvm │ ├─rhel-swap 253:4 0 10G 0 lvm │ └─rhel-opt 253:5 0 20G 0 lvm ├─sda4 8:4 0 2G 0 part /boot/efi ├─sda5 8:5 0 2G 0 part /boot └─sda6 8:6 0 30G 0 part /var
sdb 8:16 0 279.4G 0 disk ├─sdb1 8:17 0 90G 0 part │ └─rhel-root 253:1 0 100G 0 lvm └─sdb2 8:18 0 189.4G 0 part /