Track CVEs for kubernetes dependencies...

brendandburns commented 5 years ago

Kubernetes has a very large number of golang library dependencies. While there is some work to track and ensure license compatability, there is little to know work done to track vulnerabilities in these library dependencies.

Indeed, I don't know of a database (something like https://ossindex.sonatype.org/) for go libraries that we could use. (perhaps the CNCF can help here...)

But the lack of tools and databases isn't an excuse.

We need to do a better job here of tracking, reporting and updating our dependencies to fix known relevant security issues.

And ultimately, we also need to do a periodic audit to make sure that we aren't importing vulnerabilities into the codebase.

@philips @spiffxp @kubernetes/steering-committee

jayunit100 commented 5 years ago

Agree , one highly visible critical CVE unchecked could really hurt a data center and even if not, if public it would hurt the growth and adoption of kubernetes in enterprises.

Should this Be part of the build process ? We may be able to Integrate with blackduck copilot .

dims commented 5 years ago

@jayunit100 AFAICT copilot does not support golang (looking at https://copilot.blackducksoftware.com/)

cblecker commented 5 years ago

cc: @kubernetes/product-security-team

I know GitHub does this for other languages.. I wonder if they will start doing it for Go 🤔.

brendandburns commented 5 years ago

@jessfraz Do you know anything about tools that Github might be working on?

jessfraz commented 5 years ago

yes there is work on better tooling around this, so if you can hang tight for a little bit (maybe a few months but don't quote me on that) it will get better

jessfraz commented 5 years ago

I had bounced some ideas off @cblecker and the PST about our plans here with github integrations but happy to discuss with anyone else as well :) don't want to necessarily make promises in public in writing just in case things change a little but we would love feedback on the ideas going forward

brendandburns commented 5 years ago

@jessfraz any chance you can add me to the email?

jessfraz commented 5 years ago

Added!

gjohnson commented 5 years ago

In the interim, would something like dependabot work?

tallclair commented 5 years ago

A lot of our dependencies are not mature enough to get CVEs for security issues. That doesn't mean we shouldn't care about the vulnerability.
We also want to have insight into non-security bug fixes. See this slide: https://docs.google.com/presentation/d/1i3Qnt2yQOfqVLXXkyiaQ_6HOj5ZqkeWQ2wRmT1U2aBs/edit#slide=id.g499934ca8f_0_72
We should also work on trimming dependencies to reduce the number we need to worry about

bgrant0607 commented 5 years ago

On trimming dependencies: that's one of many reasons why I want provider-specific code (e.g., cloud providers, volume sources) out of tree.

philips commented 5 years ago

@brendanburns Did you ever see any response on the CNCF Service Desk request for Go dependency CVE tracking?

@tallclair Is there even a way for Go dependencies to express metadata about security right now?

tallclair commented 5 years ago

Not that I know of... Maybe it's something the go team could help us solve?

philips commented 5 years ago

@tallclair Yea, I think we are in a bit of a chicken and egg situation. If there is no way to express security issues then it is impossible to build tooling for it.

Anyone know who on the Go team to talk to? The only person I know is @bradfitz 👋

tallclair commented 5 years ago

Yea, I think we are in a bit of a chicken and egg situation.

I think there are some hacky things we could do, like creating a dashboard for commits of the dependencies. For other dependencies in github, we could also track issues & releases.

Sajmani commented 5 years ago

I'll loop in a few Go team members here; CVE tracking is definitely something we're thinking about. It would be useful to us to know what form this needs to take to integrate well with the k8s workflow.

CC @rsc @andybons @FiloSottile @julieqiu @spf13

dims commented 5 years ago

@Sajmani not sure if you have seen this already, just in case - https://github.com/kubernetes/sig-release/blob/master/security-release-process-documentation/security-release-process.md#security-release-process

caniszczyk commented 5 years ago

nothing was every filed to the CNCF servicedesk but note that FOSSA which Kubernetes uses for licensing scanning already supports security scanning

CNCF also is happy to fund other tools like Snyk that do this type of work

dims commented 5 years ago

@caniszczyk Snyk does seem interesting, let's see what others say. one issue is that they support dep/govendor and we are currently using godep

caniszczyk commented 5 years ago

@dims ya, I mean since we're already doing the FOSSA scanning for licensing checks it shouldn't be too hard to do the CVE scan too, they are more than happy to help with this, but we leave it up to the project to choose what tools work best

philips commented 5 years ago

@caniszczyk This was filed with the service desk. Maybe something is wrong with the CNCF service desk? Spam processing?

@brendanburns emailed the service desk (servicedesk@cncf.io) with the title "Request for CVE/Vulnerability tracking for golang packages." on November 28th 2018. I was cc'd on the request and sent a followup email yesterday.

On Wed, Feb 27, 2019 at 6:54 AM Chris Aniszczyk notifications@github.com wrote:

nothing was every filed to the CNCF servicedesk but note that FOSSA which Kubernetes uses for licensing scanning already supports security scanning

CNCF also is happy to fund other tools like Snyk that do this type of work

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kubernetes/community/issues/2992#issuecomment-467891886, or mute the thread https://github.com/notifications/unsubscribe-auth/AACDCN7osFvCZHOxts8UwD_FjXHunrPaks5vRpwKgaJpZM4Y5Bmu .

caniszczyk commented 5 years ago

forward me the email and I'll look into it

On Wed, Feb 27, 2019 at 11:33 AM Brandon Philips notifications@github.com wrote:

@caniszczyk This was filed with the service desk. Maybe something is wrong with the CNCF service desk? Spam processing?

@brendanburns emailed the service desk (servicedesk@cncf.io) with the title "Request for CVE/Vulnerability tracking for golang packages." on November 28th 2018. I was cc'd on the request and sent a followup email yesterday.

On Wed, Feb 27, 2019 at 6:54 AM Chris Aniszczyk notifications@github.com wrote:

nothing was every filed to the CNCF servicedesk but note that FOSSA which Kubernetes uses for licensing scanning already supports security scanning

CNCF also is happy to fund other tools like Snyk that do this type of work

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/kubernetes/community/issues/2992#issuecomment-467891886 , or mute the thread < https://github.com/notifications/unsubscribe-auth/AACDCN7osFvCZHOxts8UwD_FjXHunrPaks5vRpwKgaJpZM4Y5Bmu

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/kubernetes/community/issues/2992#issuecomment-467956398, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD5IYHUCmbjrJ7fNUWu9hb82VuTYwDrks5vRsFfgaJpZM4Y5Bmu .

-- Cheers,

Chris Aniszczyk http://aniszczyk.org +1 512 961 6719

philips commented 5 years ago

@dims @caniszczyk The problem is that the data set FOSSA and synk are working from is super duper small. There are many libraries Kubernetes depends on that we have fixed issues in that don't show up in the snyk data set.

To fix this we have to work with the Go community to enable publishing of this metadata. This is not a vendor selection issue at this point. It is a missing metadata issue.

philips commented 5 years ago

@caniszczyk Sent. For context here was the request headers.

From: Brendan Burns
Date: Wed, Nov 28, 2018, 10:35 PM
To: servicedesk@cncf.io
Subject: Request for CVE/Vulnerability tracking for golang packages.

philips commented 5 years ago

For those following along the CNCF Service desk silently ignores emails from addresses not pre-registered with it. https://twitter.com/BrandonPhilips/status/1102992489072349184

tom-snyk commented 5 years ago

@dims happy to get some of our team involved to learn your requirements! @philips appreciate this is a community publishing issue to; however, we do have some unique harvest capabilities that might be able to help source more. Let us know if you want to chat with our engineers.

fejta-bot commented 5 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

philips commented 5 years ago

/remove-lifecycle stale

cc @amye

dims commented 5 years ago

ok, first step, get dependencies to known versions (not random SHA's) - https://github.com/kubernetes/kubernetes/issues/78434

nikhita commented 5 years ago

/remove-lifecycle stale

fejta-bot commented 5 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

philips commented 5 years ago

This should be easier now that there is an API of module versions: https://index.golang.org/

tallclair commented 5 years ago

/remove-lifecycle-stale

fejta-bot commented 4 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

tallclair commented 4 years ago

/remove-lifecycle stale

fejta-bot commented 4 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

tallclair commented 4 years ago

/remove-lifecycle stale /lifecycle frozen

dims commented 4 years ago

@tom-snyk is the offer still open? :) i tried the free version and the golang scanning is not enabled there

nikhita commented 4 years ago

Update - we are talking to Snyk folks on things we'd need from them for using their tool for k/k (thanks @elson-snyk!). Conversation going on in #k8s-code-organization in slack.

LappleApple commented 3 years ago

@nikhita Checking in on this — any recent developments?

justaugustus commented 3 years ago

/assign @hasheddan @justaugustus

LappleApple commented 3 years ago

Sync with PSC and SIG Release/Rel Eng would be useful here — closer interlock would be +1

nikhita commented 3 years ago

/assign @navidshaikh

tallclair commented 3 years ago

Hopefully we can leverage https://go.googlesource.com/proposal/+/master/design/draft-vulndb.md for this in the future

BenTheElder commented 3 years ago

@tallclair +1 -- I've been in contact with the google team working on this, they've actually been using kubernetes/kubernetes as a test case given our large dependency tree 👍

PushkarJ commented 3 years ago

I am working with @navidshaikh on this from sig-security In terms of triage options of CVEs post identification regardless of tools, would love to hear some feedback on this proposal: https://docs.google.com/document/d/1V-IUJVYq4SGizNpZXA9GPBPeCJ-oH9p5V747o51NYvo/edit

Thanks @tallclair for your earlier feedback on slack for this proposal.

/sig security

BenTheElder commented 3 years ago

vulndb is now somewhat exposed in https://deps.dev/ however, it's not really setup to monitor a repo @ HEAD AFAICT. There's been some progress recently on standing up snyk I think (some recent discussion in #sig-testing, #code-organization, #wg-k8s-infra).

Sajmani commented 3 years ago

A prototype CLI for reading the Go vulndb is here: https://pkg.go.dev/golang.org/x/exp/vulndb/govulncheck This is not yet ready for production use, but we welcome early adopter feedback.

dims commented 3 years ago

@Sajmani thanks! will poke at it when we get a chance

PushkarJ commented 3 years ago

A prototype CLI for reading the Go vulndb is here: https://pkg.go.dev/golang.org/x/exp/vulndb/govulncheck This is not yet ready for production use, but we welcome early adopter feedback.

@vinayakankugoyal is this the same tool you are doing a demo about in next SIG Security Tooling meeting (08/17)?

kubernetes / community

Track CVEs for kubernetes dependencies... #2992