Progress Tracking

[joyliu-q]

Opening this issue to use it to track the progress of SASTAll.

The vision

We want a tool that is able to simplify the process of running 3 different SAST tools.

There are 2 possible solutions that would follow SASTAll's vision.

1. Publish a Github actions that is able to run 3 different SAST tools at once
2. TODO: Before, I was thinking creating a template with Helm, but that would involve K8s. I was ALSO thinking about making a docker image that just runs these tools as CLIs manually, but some of them already have pre-existing Github actions and that just seemed like doing something for the sake of doing something. If we want to do something cool with aggregation, we could host the aggregated result in the form of a SARIF file at another endpoint different from Github Securities tab, which would allow me to attempt to do something outside of CI/CD through this project.

Aggregation & Whether it's needed

Either way, the results have to be aggregated by some means. Or do they?

Before starting on this project, I thought that aggregation was one of the biggest wins/advantages of SASTAll. Running multiple tools does not change the rate of false positives (~50%), but if we can take all of the results in SARIF format and do something interesting with it, running multiple tools could actually be worth it.

However, after looking into how Github actually works, I was greatly disillusioned. Apparently for these tools, they already have CIs built into place and everything just shows up on the Github security tab. It looks great, too: there are even little tags you can filter different issues by. So, is there even a purpose to take all of the SARIF files generated, parse it, combine results, and display it somewhere?

The only advantage to doing aggregation is because currently Github just takes the issues found and throws it onto the security tab. There may be some redundancy. However, redundancy might be good because you see "Wow, all 3 of these tools did not like this particular line. Maybe I should look into this!"

But that's okay! Because here's a potential idea: do something with the Code Scanning API

TODO

• [ ] Make an MVP of a publishable github actions that—at the very minimum—runs these SAST tools.
• [ ] Store the sarif files somewhere
• [ ] ???

[joyliu-q]

For now

• [x] Add artifact upload
• [x] Allow CodeQL to save correctly similar to semgrep
• [x] Add aggregation logic as a bash script calling js script & throw that on a Dockerfile (just kidding I just realized either python or js also works)

[joyliu-q]

It worked!!

[joyliu-q]

Nice