During the review of https://github.com/OpenSCAP/openscap/pull/2051 where we introduced a limit of collected items by OpenSCAP probe we discussed in the comments also other possible approaches to limiting collected items.
The original problem that we tried to mitigate by https://github.com/OpenSCAP/openscap/pull/2051 was that the OpenSCAP failed to generate HTML report. The XSLT template that creates the HTML report failed because libxslt didn't have enough memory to process it. The reason was the input XML tree of the XSLT template was too large. That is probably caused by too many OVAL items collected.
Currently, OpenSCAP internally creates an ARF XML tree, applies XSLT on that to create HTML report, saves the HTML to a file and then saves the ARF file.
However, this limitation is present in the XSLT template, so it reduces the size of the final HTML report, but doesn't affect the internal ARF, so it doesn't prevent the fail of libxslt as described above.
The limit introduced in #2051 limits the amount of collected items by OpenSCAP probes. Therefore, it also affects output ARF, it is effective also when no HTML is created. This limit can cause incomplete or missing results depending on the specific OVAL. There was an idea by @evgenyz to try and throw out elements from the tree before trying to render them with XSLT instead of not collecting them at all. With this approach the ARF and XCCDF results would stay correct.
Due to the way how it works now it would cause a problem that also ARF file would be incomplete, which you probably don't want. But we can avoid it by creating a dedicated ARF tree just for the purpose of HTML creation, ie. first a full ARF tree would be created and saved to file and then another tree with reduced items would be created passed to XSLT to create a HTML output.
OpenSCAP Version:
1.3.9
Operating System & Version:
all
Steps to Reproduce:
This is a general issue about the logic executed during command:
evgenyz
commented
8 months ago
Description of Problem:
During the review of https://github.com/OpenSCAP/openscap/pull/2051 where we introduced a limit of collected items by OpenSCAP probe we discussed in the comments also other possible approaches to limiting collected items.
The original problem that we tried to mitigate by https://github.com/OpenSCAP/openscap/pull/2051 was that the OpenSCAP failed to generate HTML report. The XSLT template that creates the HTML report failed because libxslt didn't have enough memory to process it. The reason was the input XML tree of the XSLT template was too large. That is probably caused by too many OVAL items collected.
Currently, OpenSCAP internally creates an ARF XML tree, applies XSLT on that to create HTML report, saves the HTML to a file and then saves the ARF file.
The amount of rendered items in HTML report is limited to 100 items per OVAL object. See https://github.com/OpenSCAP/openscap/blob/c27e1085a3e211ffca970c475e880952f2207c5f/xsl/xccdf-report-oval-details.xsl#L105-L119
However, this limitation is present in the XSLT template, so it reduces the size of the final HTML report, but doesn't affect the internal ARF, so it doesn't prevent the fail of libxslt as described above.
The limit introduced in #2051 limits the amount of collected items by OpenSCAP probes. Therefore, it also affects output ARF, it is effective also when no HTML is created. This limit can cause incomplete or missing results depending on the specific OVAL. There was an idea by @evgenyz to try and throw out elements from the tree before trying to render them with XSLT instead of not collecting them at all. With this approach the ARF and XCCDF results would stay correct.
We can easily limit the amount of the items in the ARF during the serialization into OVAL results DOM. I look into that and it could be done by modifying oval_syschar_model_to_dom and oval_syschar_to_dom. https://github.com/OpenSCAP/openscap/blob/c27e1085a3e211ffca970c475e880952f2207c5f/src/OVAL/oval_sysModel.c#L360-L390
Due to the way how it works now it would cause a problem that also ARF file would be incomplete, which you probably don't want. But we can avoid it by creating a dedicated ARF tree just for the purpose of HTML creation, ie. first a full ARF tree would be created and saved to file and then another tree with reduced items would be created passed to XSLT to create a HTML output.
OpenSCAP Version:
1.3.9
Operating System & Version:
all
Steps to Reproduce:
This is a general issue about the logic executed during command:
Actual Results:
Expected Results:
Additional Information / Debugging Steps:
https://issues.redhat.com/browse/RHEL-11925, https://issues.redhat.com/browse/RHEL-4141