search

GSA / code-gov-web

DEPRECATED 🛑- Federal Source Code policy implementation.
Other
407 stars 108 forks source link

Tools for developers to help automate security and code quality #101

Open mattbailey0 opened 8 years ago

mattbailey0 commented 8 years ago

We're building out a list of developer-recommended tools that can be used to help make sure that code being pushed to public repositories is high quality, secure, complies with any relevant policies.

For example, the Consumer Financial Protection Bureau developed Clouseau to scan repos for offending patterns using regex.

Two questions: 1) What subcategories of tools should we be thinking about here? 2) What tools should we include in the list?

Tools don't necessarily have to be open source or free as in beer. They just have to be great and make developers happy.

jbjonesjr commented 8 years ago

Looking at prior art, GitHub publishes an integration directory as a way for users (and tools) to find each other and aid in the software development life cycle.

You can see the full list of Categories that GitHub defined here: https://github.com/integrations

Some that seem relevant to @mattbailey0's question for

being pushed to public repositories is high quality, secure, complies with any relevant policies

  • API Management
  • Code Quality
  • Code Review
  • Continuous Integration
  • Dependency Management
  • Open Source Management!!
  • Security

What tools should we include in the list?

I'll spare the effort of regurgitating what's in the integrations directory, but didn't want to add a bit more to the specific category Matt mentioned above about repo cleaning.

In that light, I wanted to aggregate those tools here:

mcmahonjohn commented 8 years ago

I can't recommend SonarQube enough. It is listed as an integration for GitHub. But there is also BitBucket. Which provides a lot of integrations as well: https://bitbucket.org/product/integrations

SonarQube looks at the complexity of the code and calculates a real cost in time to fix problems. Additionally, it can be expanded to view the code in terms of areas that are constantly changed and thus has greater risk of faults.

anselmbradford commented 8 years ago

CFPB has an ESLint config file in https://github.com/cfpb/front-end/ here

jfredrickson5 commented 8 years ago

@jbjonesjr Here is the proof of concept we built that you mentioned in your list. However, there are already plenty of good repo scanning apps out there such as Clouseau and Seekret. I want to refactor our tool to use one of those solutions as the scanning core of our tool. We would add on webhook support (could work with GitHub, BitBucket, GitLab, etc), and possibly tie in to the agency's code.json so that it knows which repos to scan and who to notify if an issue is found.

One important issue to point out when considering repository scanning tools is that there are really two subcategories of this, and for a complete solution, you need to implement both:

mcmahonjohn commented 8 years ago

@jfredrickson5 There are also tools that cost money to scan private repos. But since the government's goal are open source projects, then that wouldn't necessary count. But I point it out as an option for when the agency is preparing to release the code, but before it is actually public.

jbjonesjr commented 8 years ago

@jfredrickson5

I want to refactor our tool to use one of those solutions as the scanning core of our tool.

Totally agree! With so many other scanning tools being created, i've always resisted the urge to add a new one because of just this reason. Having a sort of plug-and-play architecture would be pretty cool to fill in the other gaps in the ecosystem without the scanner itself.

We would add on webhook support (could work with GitHub, BitBucket, GitLab, etc), and possibly tie in to the agency's code.json so that it knows which repos to scan and who to notify if an issue is found.

All of this is why I think what you are working on is different than what has come before it (and why I'm so excited about it). Everything else is focused on local repo-scanning, but there is still an opportunity for centralized support making it easier for maintainers to have confidence about their codebase. The static-code analysis tools out there (Fortify, SonarQube, etc) are great, but can be a bit hard to configure and maintain. There is definitely a place in the market for a turnkey, simple scanning solution deployable to the PAAS of your choice (hello ☁️.gov ).

The only other thing I would add to your list of subcategories is that tools like GitHub Enterprise also support using things like pre-commit hooks (in GitHub Enterprise's case, it's called a pre-receive hook), in that it can enforce and manage information before it is committed into the central repo. This way you don't have to rely on each contributor managing the pre-commit hooks themselves, and you can even enforce control flow as well (see examples in this repository).

jbjonesjr commented 8 years ago

@mcmahonjohn

But since the government's goal are open source projects, then that wouldn't necessary count.

I hope that the Government in this case doesn't equate open source = free or little support resources. Investing in an open source project and community includes maintaining and nourishing it (and other things discussed in #35), but shouldn't restrict from "investing" in technology and supporting tools.

Definitely want to squash that narrative!

mcmahonjohn commented 8 years ago

@jbjonesjr I was referring to the government spending money on the tools available for private repos. There will definitely be some cost before releasing the code to the public. Ideally, they would use the same tools when the repo is private for when it becomes public that would reduce the overhead on the migration.