JSON output summary incorrect

[nebc-jasonfung]

I have a custom config which sets all rules to 'warning', it appears the JSON summary output is incorrect, for example:

{
  "status": 0,
  "result": {
    "summary": {
      "flowsNumber": 1,
      "errors": 4,
      "message": "A total of \u001b[1m4\u001b[22m errors have been found in \u001b[1m1\u001b[22m flows.",
      "errorLevelsDetails": {
        "warning": 6,
        "error": 1
      }
    },
    "status": 1,
    "results": [
      {
        "flowName": "Account Update Contacts",
        "ruleName": "DMLStatementInLoop",
        "description": "To avoid hitting Apex governor limits, we recommend grouping all of your database changes together at the end of the flow, whether those changes create, update, or delete records.",
        "severity": "warning",
        "details": {
          "name": "Update_Contact",
          "type": "recordUpdates"
        }
      },
      {
        "flowName": "Account Update Contacts",
        "ruleName": "MissingFaultPath",
        "description": "Sometimes a flow doesn’t perform an operation that you configured it to do. By default, the flow shows an error message to the user and emails the admin who created the flow. However, you can control that behavior.",
        "severity": "warning",
        "details": {
          "name": "Get_Related_Contacts",
          "type": "recordLookups"
        }
      },
      {
        "flowName": "Account Update Contacts",
        "ruleName": "MissingFaultPath",
        "description": "Sometimes a flow doesn’t perform an operation that you configured it to do. By default, the flow shows an error message to the user and emails the admin who created the flow. However, you can control that behavior.",
        "severity": "warning",
        "details": {
          "name": "Update_Contact",
          "type": "recordUpdates"
        }
      },
      {
        "flowName": "Account Update Contacts",
        "ruleName": "MissingNullHandler",
        "description": "If a Get Records operation does not find any data it will return null. Use a decision element on the operation result variable to validate that the result is not null.",
        "severity": "warning",
        "details": {
          "name": "Get_Related_Contacts",
          "type": "recordLookups"
        }
      }
    ]
  }
}

errorLevelsDetails says 6 warnings and 1 error, however, the breakdown of each surfaced rule totals 4 warnings only.

I'm possibly misunderstanding how the summary attribute is calculated, so please point me in the right direction.

[RubenHalman]

Good catch and this looks like a bug. I haven't introduced the errorlevel so I will have to dive into it. For now, perhaps you can loop over the details yourself to get the number.

did you run the failon command btw?

[nebc-jasonfung]

My use case is currently trying to build a GitHub action which fails a PR if errors are found - I'm not sure loops are a thing in yaml world, so I'm currently limited to accessing a flat attribute.

I tried the following command sf flow:scan --failon error --json --config flow-scanner.json but not sure what i'm looking for as the JSON output is the same as above.

Thanks in advance for your support @RubenHalman

[RubenHalman]

@nebc-jasonfung so can you use the 'error' attribute, this one would be correct right(4)?

@nvuillam can you elaborate on what the "errorLevelsDetails" should show in this case?

[nvuillam]

  "errorLevelsDetails": {
    "warning": 6,
    "error": 1
  }

It means there are 6 issues with warning level and 1 with error level

Indeed the count seems not good... I'll have a look !

[nvuillam]

@nebc-jasonfung please could you run the same case without the --json and post the result here ?

[nebc-jasonfung]

@nvuillam, ofcourse, here are the results:

Identified 1 flows to scan... Scan complete
== Account Update Contacts ==

WARNING DMLStatementInLoop
Details: Update_Contact, recordUpdates
To avoid hitting Apex governor limits, we recommend grouping all of your database changes together at the end of the flow, whether those changes create, update, or delete records.

WARNING MissingFaultPath
Details: Get_Related_Contacts, recordLookups
Sometimes a flow doesn’t perform an operation that you configured it to do. By default, the flow shows an error message to the user and emails the admin who created the flow. However, you can control that behavior.

WARNING MissingFaultPath
Details: Update_Contact, recordUpdates
Sometimes a flow doesn’t perform an operation that you configured it to do. By default, the flow shows an error message to the user and emails the admin who created the flow. However, you can control that behavior.

WARNING MissingNullHandler
Details: Get_Related_Contacts, recordLookups
If a Get Records operation does not find any data it will return null. Use a decision element on the operation result variable to validate that the result is not null.

=== A total of 4 errors have been found in 1 flows.

it'll be good if the errorLevelsDetails contained a count of the results array where severity = 'warning' and severity = 'error'

[RubenHalman]

@nebc-jasonfung Tnis should now resolved with the latest changes by Nicolas. Thank you for your involvement.