In order to be able to predict problems such as deprecated APIs that need to be detected before new OpenShift/K8S releases we need to have controller logs available for all operators not only the ones failing or timing out. And we need that in at least two moments: after the operator install test and after the operand install test.
Operator Install Test: the operator main contain owned resources that are using a Group, Version, Kind marked for deprecation. Other warnings or errors that don't stop the operator install may also be catch there. The controller logs will show up on the operator's pod/deployment. That should be one detailed report for the operator. Ideally in a fully compliant JSON format for future database usage.
Operand Install Test: after installing an operand CRD/ALM example we may also see deprecation errors and others even if the operand ends up installing at a minimum check level, i.e., we just check if we can create the CRD/ALM example not the actual operand resources. Those are still to come on a future PR. That should be a second detailed report for the operand only.
It's interesting to have a key field that can link both operator data and operand data in order to perform queries on the data captured.
As a suggestion we can use the flag: --detailed-reports to enable them and only print out the operand one if operands are in the initial audit plan.
In order to be able to predict problems such as deprecated APIs that need to be detected before new OpenShift/K8S releases we need to have controller logs available for all operators not only the ones failing or timing out. And we need that in at least two moments: after the operator install test and after the operand install test.
Operator Install Test: the operator main contain owned resources that are using a Group, Version, Kind marked for deprecation. Other warnings or errors that don't stop the operator install may also be catch there. The controller logs will show up on the operator's pod/deployment. That should be one detailed report for the operator. Ideally in a fully compliant JSON format for future database usage.
Operand Install Test: after installing an operand CRD/ALM example we may also see deprecation errors and others even if the operand ends up installing at a minimum check level, i.e., we just check if we can create the CRD/ALM example not the actual operand resources. Those are still to come on a future PR. That should be a second detailed report for the operand only.
It's interesting to have a key field that can link both operator data and operand data in order to perform queries on the data captured.
As a suggestion we can use the flag: --detailed-reports to enable them and only print out the operand one if operands are in the initial audit plan.