Thank you for building this library and a happy holidays! This PR aims to resolve #467 and add support for outputting Stan's observations as SARIF files. To support this, I wrote a new library, sarif, which implements the relevant types to represent SARIF structures and to serialise/deserialise them to/from JSON.
The approach taken by my implementation is as follows:
I have added a dependency on the sarif library
There is a new command-line option, --sarif, which instructs Stan to produce output in the SARIF format
A new module, Stan.SARIF, contains functions to convert from Stan's representations to those used for SARIF
Once converted, the serialisation is handled by the sarif library
What's this good for?
SARIF files can be understood by other tools, such as GitHub Code Scanning. For example, I used the changes made in this PR to run Stan on itself and upload the results to Code Scanning, which will then show a list of all observations:
These can be viewed in detail:
How can I do this?
To reproduce this, simply run stan --sarif > stan.sarif and use the upload-sarif action to upload the file as part of a GitHub Actions workflow.
Questions / Points of note
The sarif library depends on aeson while stan depends on microaeson. If sarif is added as a dependency to stan, then aeson becomes a transitive dependency. Therefore, any benefits with respect to fewer dependencies obtained from using microaeson are lost. The different libraries in use here also lead to some minor complications in the integration between the two projects (e.g. with respect to writing custom properties, such as tags).
The --sarif option currently "overrides" the --json-output option (i.e. if both are specified, the output is formatted as SARIF, which sort of makes sense because SARIF is also JSON). I see a few options here:
Replace the --json-output option entirely in favour of SARIF output
Implement a check that both options are not active at the same time
Refactor the command-line options so that there are different sub-commands
... something else entirely?
I'd appreciate a review of this and your thoughts on all of the above! Thanks!
Hi there!
Thank you for building this library and a happy holidays! This PR aims to resolve #467 and add support for outputting Stan's observations as SARIF files. To support this, I wrote a new library,
sarif, which implements the relevant types to represent SARIF structures and to serialise/deserialise them to/from JSON.The approach taken by my implementation is as follows:
sariflibrary--sarif, which instructs Stan to produce output in the SARIF formatStan.SARIF, contains functions to convert from Stan's representations to those used for SARIFsariflibraryWhat's this good for?
SARIF files can be understood by other tools, such as GitHub Code Scanning. For example, I used the changes made in this PR to run Stan on itself and upload the results to Code Scanning, which will then show a list of all observations:
These can be viewed in detail:
How can I do this?
To reproduce this, simply run
stan --sarif > stan.sarifand use theupload-sarifaction to upload the file as part of a GitHub Actions workflow.Questions / Points of note
sariflibrary depends onaesonwhilestandepends onmicroaeson. Ifsarifis added as a dependency tostan, thenaesonbecomes a transitive dependency. Therefore, any benefits with respect to fewer dependencies obtained from usingmicroaesonare lost. The different libraries in use here also lead to some minor complications in the integration between the two projects (e.g. with respect to writing custom properties, such as tags).--sarifoption currently "overrides" the--json-outputoption (i.e. if both are specified, the output is formatted as SARIF, which sort of makes sense because SARIF is also JSON). I see a few options here:--json-outputoption entirely in favour of SARIF outputI'd appreciate a review of this and your thoughts on all of the above! Thanks!