Improved Battle Tests

[gotbadger]

Our original battle tests focused on testing a wide array of projects to try and find breaking bases in unusual codebases. Now we want to focus on some specific projects that will demonstrate how bearer works and performs and more importantly give us a baseline for how rules and detection improvements lead to overall better outcomes.

Projects

We will want to add more to this list as new languages are added or we found new interesting projects. Initial list looks like this:

• https://github.com/Bearer/juice-shop (TS)
• https://github.com/Bearer/railsgoat (Ruby)
• https://github.com/Bearer/WebGoat (Java)
• https://github.com/Bearer/NodeGoat (JS)
• https://github.com/forem/forem (JS)
• https://github.com/freeCodeCamp/chapter (TS)
• https://github.com/mastodon/mastodon (Ruby)
• https://github.com/frab/frab (Ruby)
• https://github.com/discourse/discourse (Ruby)
• https://github.com/diaspora/diaspora (Ruby)
• https://github.com/TryGhost/Ghost (JS)
• https://github.com/wekan/wekan (JS)
• https://gitlab.com/gitlab-org/gitlab (Ruby)
• https://github.com/grafana/grafana (TS)

Tasks

• [x] Investigate how to keep forked repos up-to-date
  • Not straight-forward. Can we just use non-forked original repos?
• [x] Set up Bearer Cloud accounts @elsapet
• [x] Add rule version to metadata that is sent during bearer scan @elsapet https://github.com/Bearer/bearer/pull/913
• [x] Create job to kick off scan process @didroe https://github.com/Bearer/bearer/pull/897
  • Q: Do we need forked repos?
• [x] Get scan data into Bearer Cloud account
• [x] Add new repositories to workflow @elsapet https://github.com/Bearer/bearer/pull/923
• [x] Set up non-testing Bearer Cloud account on Prod (Battletest) @elsapet
• [ ] Use canary builds?

[elsapet]

@gotbadger would we want an option to run the battle tests on canary builds?

[gotbadger]

@elsapet no I dont think it would fit in with our review processes anyway.