CP-48447+CA-390127: correct exception handling for func_output callbacks, fix saving all log messages

[bernhardkaindl]

Fix two issues (their tests overlap):

CA-390127 (fix saving all log messages)

Fix adding even the last log() messages to the output archive

• In the test added (extended) for CP-48447 (below) which extends test_main.py (which calls xen-bugtool.main()), the exception raised and logged as part of the test execution could not be found in the xen-bugtool.log file that is recorded in the bugtool output archive.
• However, the exception was shown to be included in the xen-bugtool.log file itself (not in the output archive)
• The reason for this behaviour was that xen-bugtool.log was collected before the exception was written to it:
• Because there is no guarantee that this file is collected last, update it to be explicitly collected at the end of processing before generating the inventory.xml and closing the archive.
• This was already approved by @GeraldEV in #93, now added to this PR for merge.

CP-48447 (correct exception handling for func_output callbacks)

A major design goal of xen-bugtool is to continue as well as possible when an error occurs. This is currently failing in a specific case when Python3 would be used to run xen-bugtool:

• Given a raised Exception with these properties:
  • raised inside a callback function declared by func_output()
  • not caught inside the callback function itself
• When logging that Exception,
  • the code misses encoding the backtrace of the exception from string to bytes when attempting to add it to the BytesIO-based file buffer to be included in the output archive.
  • The secondary exception caused is then fatal.

Traceback of an example (a real variant of caused me to notice it):

Initial exception (example):

Traceback (most recent call last):
  File "./xen-bugtool", line 724, in collect_data
    s = no_unicode(v["func"](cap))
  File "./xen-bugtool", line 1406, in dump_xapi_subprocess_info
    raise RuntimeError("error")
RuntimeError: error

This then results in a str (from the traceback) passed to io.BytesIO without encoding to bytes first, causing the 2nd, fatal Exception:

Traceback (most recent call last):
  File "./xen-bugtool", line 2399, in <module>
    sys.exit(main())
  File "./xen-bugtool", line 1316, in main
    collect_data(subdir, archive)
  File "./xen-bugtool", line 730, in collect_data
    v['output'] = StringIOmtime(s)
  File "./xen-bugtool", line 511, in __init__
    io.BytesIO.__init__(self, buf)
TypeError: a bytes-like object is required, not 'str'

That's an issue that is detected by pyright:

$ pyright xen-bugtool
xen-bugtool:730:45 - error: Argument of type "Unknown | str" cannot be assigned to parameter "buf" of type "bytes" in function "__init__"
    Type "Unknown | str" cannot be assigned to type "bytes"
      "str" is incompatible with "bytes" (reportArgumentType)

https://github.com/marketplace/actions/run-pyright-with-reviewdog https://github.com/jordemort/action-pyright

[codecov[bot]]

Codecov Report

Merging #92 (60c73a5) into master (d0e569d) will increase coverage by 3%. Report is 11 commits behind head on master. The diff coverage is 100%.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #92 +/- ## ===================================== + Coverage 88% 91% +3% ===================================== Files 18 19 +1 Lines 2196 2202 +6 ===================================== + Hits 1939 2011 +72 + Misses 257 191 -66 ``` | [Files](https://app.codecov.io/gh/xenserver/status-report/pull/92?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xenserver) | Coverage Δ | | |---|---|---| | [tests/unit/test\_main.py](https://app.codecov.io/gh/xenserver/status-report/pull/92?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xenserver#diff-dGVzdHMvdW5pdC90ZXN0X21haW4ucHk=) | `100% <100%> (ø)` | | | [tests/unit/test\_output.py](https://app.codecov.io/gh/xenserver/status-report/pull/92?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xenserver#diff-dGVzdHMvdW5pdC90ZXN0X291dHB1dC5weQ==) | `100% <100%> (ø)` | | | [xen-bugtool](https://app.codecov.io/gh/xenserver/status-report/pull/92?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xenserver#diff-eGVuLWJ1Z3Rvb2w=) | `88% <100%> (+3%)` | :arrow_up: | | [Flag](https://app.codecov.io/gh/xenserver/status-report/pull/92/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xenserver) | Coverage Δ | | |---|---|---| | [python2.7](https://app.codecov.io/gh/xenserver/status-report/pull/92/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xenserver) | `89% <100%> (+3%)` | :arrow_up: | | [python3.10.13](https://app.codecov.io/gh/xenserver/status-report/pull/92/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xenserver) | `91% <100%> (+3%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=xenserver#carryforward-flags-in-the-pull-request-comment) to find out more.