oasis-tcs / sarif-spec

OASIS SARIF TC: Repository for development of the draft standard, where requests for modification should be made via Github Issues
https://github.com/oasis-tcs/sarif-spec
Other
168 stars 47 forks source link

Consider: Remove Dynamic specific properties and terminology #43

Closed DerSaidin closed 7 years ago

DerSaidin commented 7 years ago

Properties such as threadId are only relevant to dynamic analysis. Should they be first class properties in this format, or left for users to put in catch-all property bags?

Also, I think one could argue the whole stack object and stackFrame object are both dynamic/runtime concepts. The stackFrame having properties like threadId and address also suggests these structures are for dynamic analysis results.

I'd also note, as far as I'm aware, the terms "stack" and "stack frame" only refer to runtime datastructures. I think it would be inaccurate to use them as a way of presenting the static concept of calling context.

michaelcfanning commented 7 years ago

The distinction here is around the production technique, not the information bundled in a result. A static analysis tool can produce a speculative vulnerability by modeling the creation of threads, emulating code execution, etc. It is useful to describe these results with the presumed runtime conditions where an error occurs. A dynamic analysis tool, of course, can capture the same information (in this case, reflecting actual conditions when code is executed).

The line we've drawn between static and dynamic analysis has mostly to do with distinctions around what's required to invoke/describe tool runs and results. A web endpoint analyzer, for example, might have significant configuration that's not common to a static analysis scenario (a root URL to start crawling, a user agent to provide during crawl, a user setting that indicates the crawler should restrict itself to the domain and subdomain of the starting point, etc.). The results of a dynamic checker tend to be pull in additional data (a fuzzer may collect a memory dump on crash, a web page checker might capture a rendered version of a page for the user to see, etc.).

In the past, some have pointed out that dynamic analysis is a kind of runtime testing and argue that dynamic analysis results should be bundled with a common format covering test execution in general. There's merit to this perspective.

All the above, and practical considerations around time/focus, has led us to the following general principle, captured in the Sarif Design Principles document. If you'd like to propose any changes to this, definitely put them forward.

5.SARIF is designed for static analysis tools and any concept that generally applies for this scenario shall be considered for the format. SARIF can clearly be used for many dynamic analysis scenarios and we should consider augmenting the format for this class of tooling, but not in cases where what is proposed is applicable to the dynamic analysis domain only.

michaelcfanning commented 7 years ago

Closing this issue for now. @DerSaidin, please reactivate if you'd like to continue discussion.