search

redhat-cop / infra.leapp

Collection of Ansible roles for automating RHEL in-place upgrades using Leapp.
MIT License
43 stars 33 forks source link

Improve error reporting. #169

Closed jeffmcutter closed 3 months ago

jeffmcutter commented 3 months ago

Display inhibitors and errors encountered in analysis and upgrade analysis. Capture errors not marked as such in json report, only in txt report.

jeffmcutter commented 3 months ago

Sample output for analysis with high (error):

RUNNING HANDLER [infra.leapp.analysis : Display inhibitors] ********************
skipping: [satrhel7-2.localdomain.local]
Tuesday 26 March 2024  03:43:02 +0000 (0:00:00.030)       0:02:57.641 ********* 
 [started HANDLER: infra.leapp.analysis : Display errors on satrhel7-2.localdomain.local]

RUNNING HANDLER [infra.leapp.analysis : Display errors] ************************
ok: [satrhel7-2.localdomain.local] => {
    "results_errors.stdout_lines": [
        "Risk Factor: high (error)",
        "Title: Not enough space available on /var/lib/leapp/scratch: Needed at least 4352 MiB.",
        "Summary: {\"detail\": \"The file system hosting the /var/lib/leapp/scratch directory does not contain enough free space to proceed all parts of the in-place upgrade. Note the calculated required free space is the minimum derived from upgrades of minimal systems and the actual needed free space could be higher.\\nNeeded at least: 4352 MiB.\\nSuggested free space: 5352 MiB (or more).\"}",
        "Key: c60f42f51eed1da6788ccce8be4327ec43a1690c",
        "----------------------------------------"
    ]
}
Tuesday 26 March 2024  03:43:02 +0000 (0:00:00.035)       0:02:57.676 ********* 
 [started HANDLER: infra.leapp.analysis : Preupgrade analysis report is done on satrhel7-2.localdomain.local]

RUNNING HANDLER [infra.leapp.analysis : Preupgrade analysis report is done] ****
ok: [satrhel7-2.localdomain.local] => {
    "msg": "The preupgrade analysis report generation is now complete. WARNING: Inhibitors found. Review the tasks above or the result file at /var/log/leapp/leapp-report.txt."
}

PLAY RECAP *********************************************************************
satrhel7-2.localdomain.local : ok=35   changed=9    unreachable=0    failed=0    skipped=8    rescued=0    ignored=0

Sample output for analysis with inhibitor:

RUNNING HANDLER [infra.leapp.analysis : Display inhibitors] ********************
ok: [satrhel8-2.localdomain.local] => {
    "results_inhibitors.stdout_lines": [
        "Risk Factor: high (inhibitor)",
        "Title: Possible problems with remote login using root account",
        "Summary: OpenSSH configuration file will get updated to RHEL9 version, no longer allowing root login with password. It is a good practice to use non-root administrative user and non-password authentications, but if you rely on the remote root login, this change can lock you out of this system.",
        "Remediation: [hint] If you depend on remote root logins using passwords, consider setting up a different user for remote administration or adding a comment into the sshd_config next to the \"PermitRootLogin yes\" directive to prevent rpm replacing it during the upgrade.",
        "Key: 3d21e8cc9e1c09dc60429de7716165787e99515f",
        "----------------------------------------"
    ]
}
Tuesday 26 March 2024  03:42:28 +0000 (0:00:00.033)       0:02:47.896 ********* 
 [started HANDLER: infra.leapp.analysis : Display errors on satrhel8-2.localdomain.local]

RUNNING HANDLER [infra.leapp.analysis : Display errors] ************************
skipping: [satrhel8-2.localdomain.local]
Tuesday 26 March 2024  03:42:28 +0000 (0:00:00.032)       0:02:47.928 ********* 
 [started HANDLER: infra.leapp.analysis : Preupgrade analysis report is done on satrhel8-2.localdomain.local]

RUNNING HANDLER [infra.leapp.analysis : Preupgrade analysis report is done] ****
ok: [satrhel8-2.localdomain.local] => {
    "msg": "The preupgrade analysis report generation is now complete. WARNING: Inhibitors found. Review the tasks above or the result file at /var/log/leapp/leapp-report.txt."
}

PLAY RECAP *********************************************************************
satrhel8-2.localdomain.local : ok=33   changed=9    unreachable=0    failed=0    skipped=10   rescued=0    ignored=0

Failed upgrade due to high (error):

TASK [infra.leapp.upgrade : leapp-upgrade | Display inhibitors] ****************
skipping: [satrhel7-2.localdomain.local]
Tuesday 26 March 2024  04:19:51 +0000 (0:00:00.033)       0:02:23.421 ********* 
 [started TASK: infra.leapp.upgrade : leapp-upgrade | Display errors on satrhel7-2.localdomain.local]

TASK [infra.leapp.upgrade : leapp-upgrade | Display errors] ********************
ok: [satrhel7-2.localdomain.local] => {
    "results_errors.stdout_lines": [
        "Risk Factor: high (error)",
        "Title: Not enough space available on /var/lib/leapp/scratch: Needed at least 4352 MiB.",
        "Summary: {\"detail\": \"The file system hosting the /var/lib/leapp/scratch directory does not contain enough free space to proceed all parts of the in-place upgrade. Note the calculated required free space is the minimum derived from upgrades of minimal systems and the actual needed free space could be higher.\\nNeeded at least: 4352 MiB.\\nSuggested free space: 5352 MiB (or more).\"}",
        "Key: c60f42f51eed1da6788ccce8be4327ec43a1690c",
        "----------------------------------------"
    ]
}
Tuesday 26 March 2024  04:19:51 +0000 (0:00:00.033)       0:02:23.454 ********* 
 [started TASK: infra.leapp.upgrade : leapp-upgrade | Fail Leapp upgrade on satrhel7-2.localdomain.local]

TASK [infra.leapp.upgrade : leapp-upgrade | Fail Leapp upgrade] ****************
fatal: [satrhel7-2.localdomain.local]: FAILED! => {"changed": false, "msg": "Errors encountered running Leapp upgrade command. Review the tasks above or the result file at /var/log/leapp/leapp-report.txt."}
Tuesday 26 March 2024  04:19:51 +0000 (0:00:00.032)       0:02:23.486 ********* 
 [started HANDLER: infra.leapp.common : Add end time to log file on satrhel7-2.localdomain.local]

RUNNING HANDLER [infra.leapp.common : Add end time to log file] ****************
changed: [satrhel7-2.localdomain.local]
Tuesday 26 March 2024  04:19:52 +0000 (0:00:00.432)       0:02:23.918 ********* 
 [started HANDLER: infra.leapp.common : Slurp ripu.log file on satrhel7-2.localdomain.local]

RUNNING HANDLER [infra.leapp.common : Slurp ripu.log file] *********************
ok: [satrhel7-2.localdomain.local]
Tuesday 26 March 2024  04:19:52 +0000 (0:00:00.262)       0:02:24.181 ********* 
 [started HANDLER: infra.leapp.common : Decode ripu.log file on satrhel7-2.localdomain.local]

RUNNING HANDLER [infra.leapp.common : Decode ripu.log file] ********************
ok: [satrhel7-2.localdomain.local]
Tuesday 26 March 2024  04:19:52 +0000 (0:00:00.031)       0:02:24.212 ********* 
 [started HANDLER: infra.leapp.common : Rename log file on satrhel7-2.localdomain.local]

RUNNING HANDLER [infra.leapp.common : Rename log file] *************************
changed: [satrhel7-2.localdomain.local]

PLAY RECAP *********************************************************************
satrhel7-2.localdomain.local : ok=31   changed=4    unreachable=0    failed=1    skipped=10   rescued=1    ignored=0

Failed upgrade due to inhibitors:

TASK [infra.leapp.upgrade : leapp-upgrade | Display inhibitors] ****************
ok: [satrhel8-2.localdomain.local] => {
    "results_inhibitors.stdout_lines": [
        "Risk Factor: high (inhibitor)",
        "Title: Possible problems with remote login using root account",
        "Summary: OpenSSH configuration file will get updated to RHEL9 version, no longer allowing root login with password. It is a good practice to use non-root administrative user and non-password authentications, but if you rely on the remote root login, this change can lock you out of this system.",
        "Remediation: [hint] If you depend on remote root logins using passwords, consider setting up a different user for remote administration or adding a comment into the sshd_config next to the \"PermitRootLogin yes\" directive to prevent rpm replacing it during the upgrade.",
        "Key: 3d21e8cc9e1c09dc60429de7716165787e99515f",
        "----------------------------------------"
    ]
}
Tuesday 26 March 2024  04:21:01 +0000 (0:00:00.029)       0:03:23.226 ********* 
 [started TASK: infra.leapp.upgrade : leapp-upgrade | Display errors on satrhel8-2.localdomain.local]

TASK [infra.leapp.upgrade : leapp-upgrade | Display errors] ********************
skipping: [satrhel8-2.localdomain.local]
Tuesday 26 March 2024  04:21:01 +0000 (0:00:00.025)       0:03:23.252 ********* 
 [started TASK: infra.leapp.upgrade : leapp-upgrade | Fail Leapp upgrade on satrhel8-2.localdomain.local]

TASK [infra.leapp.upgrade : leapp-upgrade | Fail Leapp upgrade] ****************
fatal: [satrhel8-2.localdomain.local]: FAILED! => {"changed": false, "msg": "Errors encountered running Leapp upgrade command. Review the tasks above or the result file at /var/log/leapp/leapp-report.txt."}
Tuesday 26 March 2024  04:21:01 +0000 (0:00:00.049)       0:03:23.302 ********* 
 [started HANDLER: infra.leapp.common : Add end time to log file on satrhel8-2.localdomain.local]

RUNNING HANDLER [infra.leapp.common : Add end time to log file] ****************
changed: [satrhel8-2.localdomain.local]
Tuesday 26 March 2024  04:21:01 +0000 (0:00:00.306)       0:03:23.608 ********* 
 [started HANDLER: infra.leapp.common : Slurp ripu.log file on satrhel8-2.localdomain.local]

RUNNING HANDLER [infra.leapp.common : Slurp ripu.log file] *********************
ok: [satrhel8-2.localdomain.local]
Tuesday 26 March 2024  04:21:01 +0000 (0:00:00.237)       0:03:23.845 ********* 
 [started HANDLER: infra.leapp.common : Decode ripu.log file on satrhel8-2.localdomain.local]

RUNNING HANDLER [infra.leapp.common : Decode ripu.log file] ********************
ok: [satrhel8-2.localdomain.local]
Tuesday 26 March 2024  04:21:01 +0000 (0:00:00.029)       0:03:23.875 ********* 
 [started HANDLER: infra.leapp.common : Rename log file on satrhel8-2.localdomain.local]

RUNNING HANDLER [infra.leapp.common : Rename log file] *************************
changed: [satrhel8-2.localdomain.local]

PLAY RECAP *********************************************************************
satrhel8-2.localdomain.local : ok=27   changed=4    unreachable=0    failed=1    skipped=14   rescued=1    ignored=0

Inhibitor added to leapp_report_json.entries:

            {
                "actor": "target_userspace_creator",
                "audience": "sysadmin",
                "flags": [],
                "hostname": "satrhel7-2.localdomain.local",
                "id": "62c1fb3887d4317477bfbee435a59ff01067a7e1959741df3cf575d69286f324",
                "key": "c60f42f51eed1da6788ccce8be4327ec43a1690c",
                "severity": "high",
                "summary": "{\"detail\": \"The file system hosting the /var/lib/leapp/scratch directory does not contain enough free space to proceed all parts of the in-place upgrade. Note the calculated required free space is the minimum derived from upgrades of minimal systems and the actual needed free space could be higher.\\nNeeded at least: 4352 MiB.\\nSuggested free space: 5352 MiB (or more).\"}",
                "tags": [],
                "timeStamp": "2024-03-26T04:29:07.485905Z",
                "title": "Not enough space available on /var/lib/leapp/scratch: Needed at least 4352 MiB."
            },
            {
                "audience": "sysadmin",
                "flags": [
                    "inhibitor"
                ],
                "hostname": "satrhel7-2.localdomain.local",
                "severity": "high",
                "summary": "Risk Factor: high (error)\nTitle: Not enough space available on /var/lib/leapp/scratch: Needed at least 4352 MiB.\nSummary: {\"detail\": \"The file system hosting the /var/lib/leapp/scratch directory does not contain enough free space to proceed all parts of the in-place upgrade. Note the calculated required free space is the minimum derived from upgrades of minimal systems and the actual needed free space could be higher.\\nNeeded at least: 4352 MiB.\\nSuggested free space: 5352 MiB (or more).\"}\nKey: c60f42f51eed1da6788ccce8be4327ec43a1690c\n----------------------------------------",
                "title": "Likely issues found with high (error) severity found, see summary."
            }
jeffmcutter commented 3 months ago

No impact on 6-7:

RUNNING HANDLER [infra.leapp.analysis : Display inhibitors] ********************
skipping: [satrhel6-2.localdomain.local]
Tuesday 26 March 2024  03:43:24 +0000 (0:00:00.029)       0:04:00.077 ********* 
 [started HANDLER: infra.leapp.analysis : Display errors on satrhel6-2.localdomain.local]

RUNNING HANDLER [infra.leapp.analysis : Display errors] ************************
skipping: [satrhel6-2.localdomain.local]
Tuesday 26 March 2024  03:43:24 +0000 (0:00:00.015)       0:04:00.093 ********* 
 [started HANDLER: infra.leapp.analysis : Preupgrade analysis report is done on satrhel6-2.localdomain.local]

RUNNING HANDLER [infra.leapp.analysis : Preupgrade analysis report is done] ****
ok: [satrhel6-2.localdomain.local] => {
    "msg": "The preupgrade analysis report generation is now complete. SUCCESS: No inhibitors found. Review the tasks above or the result file at /root/preupgrade/result.txt."
}

PLAY RECAP *********************************************************************
satrhel6-2.localdomain.local : ok=30   changed=14   unreachable=0    failed=0    skipped=8    rescued=0    ignored=0
jeffmcutter commented 3 months ago

Fixed merge conflict and retested.