Configuring sarif reports/ Azure Pipelines Scans

[Lboer]

I have set up Robocop with Azure Pipelines for a Robot Framework repository. I am generating a sarif report and am publishing it as a build artifact, so it shows up on the Scans tab of Azure Pipelines.

- script: |
    python -m robocop --reports sarif .
    python -m robocop --reports all --list-reports
  displayName: "Run Robocop Linter"

- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: '$(Build.SourcesDirectory)/.sarif.json'
    artifactName: CodeAnalysisLogs
  displayName: "Build Artifact"

This all runs without issue, however I would like to see more data in the Scans tab. When I download the report, it includes what line the problem is on, but I don't see the lines showing up in the Scans tab.

I know I can write a custom script to parse over all sarif entries with a location and edit the Message.Text to get it to show up in the message.text, but is there a way to configure the report or Azure to include the location line in the Scans tab?

Here's a sarif warning entry for context.

"ruleId": "0310",
"level": "warning",
"message": {
    "text": "Test, suite and global variables should be uppercase"
},
"locations": 
[
     {
        "physicalLocation": {
        "artifactLocation": {
            "uri": "RobotFrameworkApiPackage/database-keywords.resource",
            "uriBaseId": "%SRCROOT%"
        },
        "region": {
            "startLine": 113,
             "endLine": 113,
            "startColumn": 27,
            "endColumn": 38
            }
        }
    }
]

[bhirsz]

As you see, the sarif report contains 'region' with startLine. That's why it should be parsed by your tool/platform and showed - if it's not happening, it's likely issue with Azure Pipelines itself (or the plugin you're using). For instance, Github parses the same Sarif report and shows the lines both in general and detail view:

But the Sarif format is extensive so I could miss additional properties. Can you show example how do you expect the report to looks like?

Maybe the artfactLocation doesn't resolve correctly and prevents scan tab to create links to file.

[Lboer]

I want the report to either have an extra tab that says "line", or I want the Details tab to give the description and finish it with "at line x".

I'm pretty sure that my Azure Devops uses this https://marketplace.visualstudio.com/items?itemName=sariftools.scans extension for the Scans tab, so I'll be creating an issue at that github to ask for documentation. I will keep you posted in this thread if something comes from that inquiry.

[bhirsz]

Ok. The easiest workaround for now would be create custom reporter (based on Sarif one) and just modify message.text attribute to contain 'at line {}' suffix.

For example here: https://github.com/MarketSquare/robotframework-robocop/blob/da977c3b14d982a631c5067bc237db0ec366bc6b/robocop/reports/sarif_report.py#L73

it would need to be something along ``f"{issue.desc} at line {issue.line}". Such custom reporter would need to be stored in your repository and called when running robocop.

And for final solution we would need to see what we can do with accordance to Sarif format. I don't want to break any tooling by adding attributes that are not supported. But if they are (and AzureDevops tool devs confirm which ones) we can additionaly update original Sarif report.