search

kubernetes / sig-security

Process documentation, non-code deliverables, and miscellaneous artifacts of Kubernetes SIG Security
Apache License 2.0
166 stars 55 forks source link

Create a periodically auto-refreshing list of fixed CVEs #1

Open PushkarJ opened 3 years ago

PushkarJ commented 3 years ago

With growing number of eyes on Kubernetes, the number of CVEs related to Kubernetes have increased. Although most CVEs are regularly fixed that directly or indirectly or transitively impact Kubernetes, there is no single place to programmatically subscribe or pull the data of fixed CVEs, for the end users of Kubernetes.

Current State of the Art

All these options are broken or incomplete:

  1. RSS feed with google groups is broken: https://github.com/kubernetes/website/issues/29142
  2. CVEDetails website seems to have incomplete data, with missing CVEs from 2021 and no mention of CVEs in base image or build time deps.
  3. This page: https://kubernetes.io/docs/reference/issues-security/issues/ links to a Github issue filter for CVE related fixes but is a broad search term

Metadata

Pre-requisites

Implementation Details

https://github.com/kubernetes/enhancements/tree/master/keps/sig-security/3203-auto-refreshing-official-cve-feed

TestGrid for GCS Bucket is available here: https://testgrid.k8s.io/sig-security-cve-feed#auto-refreshing-official-cve-feed

Optional: Trigger k/website rebuild using netlify build-hook

Beta to GA Graduation Scope

### Alpha to Beta Graduation Scope
- [x] https://github.com/kubernetes/sig-security/issues/77
- [x] https://github.com/kubernetes/sig-security/issues/73
- [x] https://github.com/kubernetes/sig-security/issues/71
- [x] https://github.com/kubernetes/sig-security/issues/72
- [x] https://github.com/kubernetes/website/issues/36808
- [x] https://github.com/kubernetes/sig-security/issues/63
### Feedback since `beta` that is resolved
- [ ] https://github.com/kubernetes/sig-security/issues/97
- [ ] https://github.com/kubernetes/kubernetes/issues/118437
- [ ] https://github.com/kubernetes/sig-security/pull/92
- [ ] https://github.com/kubernetes/sig-security/pull/106
- [ ] https://github.com/kubernetes/sig-security/issues/85
- [ ] https://github.com/kubernetes/test-infra/pull/31076

Feedback received but that requires more engagement and participation

Related Discussions

cc @sftim @tallclair @kubernetes/sig-security-leads @raesene

/committee product-security /sig security docs release

coderanger commented 3 years ago

Isn't this the problemspace https://osv.dev/ exists for? :)

PushkarJ commented 3 years ago

/assign @PushkarJ

PushkarJ commented 3 years ago

@coderanger https://osv.dev/ seems like a cool project, I did not know about this before :) I tried searching for kubernetes there and found one result. Maybe potential outcome of this exercise is a database (generated JSON doc) that can be consumed by https://osv.dev/ so users can use it to find out if their kubernetes version is impacted by any CVE or not.

PushkarJ commented 3 years ago

/transfer sig-security

sftim commented 3 years ago

generated JSON doc

We can almost certainly also consume that through Hugo and render a summary on https://k8s.io/

PushkarJ commented 3 years ago

@tabbysable @tallclair as SIG Security and SRC members, can you please confirm that you are in favor of this feature by commenting +1 to this issue. The progress on this issue is currently blocked, because of missing written consensus from SIG-Security and SRC as per this comment

PushkarJ commented 2 years ago

Just for everyone keeping track of this issue: We got a go ahead for starting work on this idea as KEP after merging the pre-requisite PR: https://github.com/kubernetes/test-infra/pull/23428 . All the linked issues are coming from this filter.

Request @tabbysable and other SRC members to add / remove the label on anything we missed. The in-scope issues are the closed issues for which there is a CVE ID and is officially announced as a Kubernetes CVE by SRC in the past. Also, for any future such issues, please add this label so it will automatically get picked up by the feed!

PushkarJ commented 2 years ago

/assign @nehaLohia27

(She is going to get the ball rolling on the KEP)

tabbysable commented 2 years ago

PR is open to update SRC documentation: https://github.com/kubernetes/committee-security-response/pull/133

sbs2001 commented 2 years ago

FWIW I've been converting the security announcements to yaml format as part of ismyk8ssecure. See advisories in particular. It contains the CVE and a list of versions of the particular kubernetes component which is vulnerable to it.

We should be able to add the first patched version, fairly easily.

sftim commented 2 years ago

Would anyone like help on this? I can try to provide advice on next steps.

sbs2001 commented 2 years ago

@sftim I would be happy to help. Let me know the next steps.

sftim commented 2 years ago

Here's the key / blocking challenge:

Create a Prow job to periodically generate this JSON document

if someone's picking this up who knows Prow: great! Feel free to get started on work to generate that document and publish it. if you're new to Prow but would like help, SIG Testing are the folks who can ask. It's also OK to reply here to ask for advice.

(personally, I don't have much experience with Prow)

PushkarJ commented 2 years ago

@sbs2001 As Tim suggested the prow job work is the main piece that needs help. @nehaLohia27 has a lot of context on this issue (and a little on prow) so please connect with her as well if you start exploring this.

oliverchang commented 2 years ago

Hi there! https://osv.dev maintainer here.

We'd love for Kubernetes to be able to publish its feed of vulnerabilities in our machine readable OSV format. We developed this format in collaboration with many in the open source community, including GitHub. This has so far been adopted by many other open source vulnerability feeds.

Our goal with this format was to make it:

I'd be very happy to work with the folks involved here to work out the details together. All that's really involved is to make this feed available in the OSV format as JSON files in either a git repo or cloud bucket.

sftim commented 2 years ago

OSV format feed

Sounds reasonable. Maybe some other project already has tooling we can borrow ideas from. Also, if we want to publish the equivalent data in another format as well one day, that should be feasible (we don't have to settle on OSV only).

PushkarJ commented 2 years ago

Agree with the suggestion on OSV. I have added a to-do to support OSV format. Once the data munging is ready, will let everyone know so we can start publishing the CVEs in OSV, JSON and maybe other formats too.

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

PushkarJ commented 2 years ago

/remove-lifecycle stale

(KEP is in progress)

PushkarJ commented 2 years ago

Alpha (in-scope of v1.25) implementation of Auto-refreshing CVE feed is now complete. In other words the feed should be available for consumption once v1.25 is GA.

Feature blog describing this feature is in progress.

oliverchang commented 2 years ago

Awesome to hear! Is there any way we can help with getting OSV support for this?

Are there any details on how the CVE JSON feeds are generated?

PushkarJ commented 2 years ago

OSV support is planned for beta stage.

For more details on current implementation please check this: https://github.com/kubernetes/enhancements/issues/3203

PushkarJ commented 2 years ago

:sparkles: Kubernetes v1.25 is live :sparkles:

What that means is that the official CVE feed (feature state: alpha) built as part of KEP-3203 is live too. You can find it here:

Upcoming blog posts to be published on Sept 12 will cover more details

angusm43ge commented 2 years ago

This is a great idea. One observation about the current alpha version -I picked a few CVEs at random from the list, and when I got into the detail, saw that they were found and resolved in older versions (for ex. 1.5.1). Support for OSV, which I see is planned for the beta, should sort that out, as the "affected versions" etc items in the schema capture that : https://ossf.github.io/osv-schema/#affectedversions-field

kevincox commented 2 years ago

The CVE feed is not valid: https://validator.jsonfeed.org/?url=https%3A%2F%2Fkubernetes.io%2Fdocs%2Freference%2Fissues-security%2Fofficial-cve-feed%2Findex.json

Is there any tracking work to ensure that the feed is valid? The most notable issue is that one of the entries (https://github.com/kubernetes/kubernetes/issues/91507) has a null ID but all of the entries are also invalid because they contain no content. Maybe a simple <a> tag linking to the GitHub issue should be added as content?

PushkarJ commented 2 years ago

Thanks @angusm43ge that is good point. It is going to take some work to have meaningful automation to detect affected versions, but we will mark it as a future feature request along with osv like you pointed out.

PushkarJ commented 2 years ago

@kevincox thanks for making me aware that a validator for JSON feed spec exists - I was not aware of this.

We will consider in future, making the feed support the spec fully or move away from the spec with our own modifications that serve the purpose for this feed.

kevincox commented 2 years ago

Thanks. I would highly recommend updating to match the spec because it opens a wide set of preexisting tooling for things like automatically notifying people when a new item is posted. But of course if for some reason you don't want to it is definitely better to completely move away and stop pretending than the current "inspired by" version.

sftim commented 2 years ago

The CVE feed is not valid: https://validator.jsonfeed.org/?url=https%3A%2F%2Fkubernetes.io%2Fdocs%2Freference%2Fissues-security%2Fofficial-cve-feed%2Findex.json

Is there any tracking work to ensure that the feed is valid? The most notable issue is that one of the entries (kubernetes/kubernetes#91507) has a null ID but all of the entries are also invalid because they contain no content. Maybe a simple <a> tag linking to the GitHub issue should be added as content?

@kevincox would you be willing to file this as an issue against k/website specifically?

sftim commented 2 years ago

We also need work to make sure that the data that the website consumes are valid too.

kevincox commented 2 years ago

Sure, filed https://github.com/kubernetes/website/issues/36808.

sftim commented 2 years ago

Include metadata about guarantees of freshness in the JSON feed: a) Prow job link b) last updated date

https://github.com/kubernetes/sig-security/issues/63 is a feature request on the same topic

krol3 commented 2 years ago

Excellent feature! Congrats to the team, we could add more information about the impacted component, for example in the PRs we have more information that we could add in the json, for example:

mtardy commented 1 year ago

The fixes that require adding "fresh" fields at the root of the object like:

would need to actually output the whole object from the script, thus removing the static part from the website: https://github.com/kubernetes/website/blob/cafe6d258c91c3814d83c0655c8c6354e3eade1c/layouts/_default/cve-feed.json

We will need some synchronization during the merge.

I created the following PRs that (if correct) should be merged sensibly at the same time, first the ones from k/sig-security then k/website:

mtardy commented 1 year ago

For the update, we merged:

So now the CVE feed is JSON feed compliant, RSS compliant and has a top level last_updated field.

k8s-triage-robot commented 1 year ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

PushkarJ commented 1 year ago

Still working on this, until we are GA; currently at beta

/remove-lifecycle stale

k8s-triage-robot commented 7 months ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

PushkarJ commented 7 months ago

Exciting updates coming soon

/remove-lifecycle stale

andrewpollock commented 1 month ago

Hi @PushkarJ could you provide an update on where things are at here?