search

oasis-tcs / sarif-spec

OASIS SARIF TC: Repository for development of the draft standard, where requests for modification should be made via Github Issues
https://github.com/oasis-tcs/sarif-spec
Other
165 stars 47 forks source link

SARIF 2.2 Proposal: `workItemUris` no longer required to be all or nothing for a run #614

Open ShiningMassXAcc opened 10 months ago

ShiningMassXAcc commented 10 months ago

Per https://docs.oasis-open.org/sarif/sarif/v2.1.0/errata01/os/sarif-v2.1.0-errata01-os-complete.html#_Toc141790915:

The workItemUris values for all result objects in theRun SHALL be either all null or all non-null.

NOTE 1: The rationale is that an engineering system will generally track work item status for all results or for none of them. Requiring that the workItemUris values be either all null or all non-null enables a consumer to determine whether work item information is available for the run by examining a single result object.

This rationale has not been true for our teams' use cases. In particular, work item tracking will exist as configured by the customer within the result tracking system for a subset of results based on those results properties. For example, a tool may only file work items for results of level error.

Additionally, a log can contain multiple runs and multiple tools - those runs and tools may not have the same treatment by the ES tool doing the work item management.

Included in this, we should not encourage the examination of one result in a run to determine overall presence of these items.

Thoughts?

Note - I'm not beholden to this being included in 2.2, but using that for consistent titling for now.

KalleOlaviNiemitalo commented 10 months ago

suppressions is likewise required to be all or nothing. Related issues: https://github.com/dotnet/roslyn/issues/62894, https://github.com/microsoft/sarif-sdk/issues/2508.

But perhaps the rationale for allowing sparsely specified workItemUris does not apply to suppressions.

michaelcfanning commented 9 months ago

Yes, suppressions is a special case, imagine a log file with no explicit empty suppressions. Was this because there weren't, in fact, any suppressions? Or did the user perhaps fail to compile code conditionally (as is required in .NET) to obtain the suppressions data?

We followed a similar line of thinking with work items, i.e., does the absence of any work item data mean a work item hasn't been filed yet? Or does it mean that the log file we're processing is work items-agnostic?