search

StatCan / aaw

Documentation for the Advanced Analytics Workspace Platform
https://statcan.github.io/aaw/
Other
68 stars 12 forks source link

Notebook Security Scanning: Runtime Scanning 2: Electric Boogaloo #923

Closed Jose-Matsuda closed 2 years ago

Jose-Matsuda commented 2 years ago

EPIC https://github.com/StatCan/daaas/issues/461

Deliverable

Have the image running as a cronjob on dev.

This now relies on us getting Artifactory to proxy ACR successfully, of which I don't have the know-how or expertise on how to solve it (probably don't have permissions either)

A lot of groundwork has been done in the public repository of mine. This was also created under the assumption that we would move our images in the acr to a local repository in Artifactory so JFROG XRAY could just run nightly scans. Discuss this at Tech elab tomorrow,

Scope of this is just getting the list of images and checking their vulnerability

Extra Info: 1) We are keeping ACR for the time being. Seems like an option is to proxy the ACR and just use the remote repository option. 2) I have a general script that retrieves a list of notebook images used in the cluster as well as images throughout the pods so can use that when deciding what images I need to scan.

Tasks

If we go with trivy we will need to download the image from the ACR and then run the trivy on it, and then parse through the output. If we use JFROG XRAY, assuming we use a remote repository, then the situation may be to try and pull images from a remote-repository to trigger a scan. So either way, as a result of not having them stored locally some pull will need to happen for scans to happen

Arguments for using Artifactory as a proxy.

1) Probably the biggest one, as Blair mentioned, it would be easier for us to handle and create exceptions in the 'scanning' rules as that is supported by XRAY.

Decision

Move forward with Artifactory proxying ACR

IF we choose to leverage Artifactory

https://github.com/StatCan/daaas/issues/933

Adapt script in response to the above decision to actually do the scanning.

Work will be hosted in my personal repo (from above under 2022) Steps

Jose-Matsuda commented 2 years ago

image

and ofc on a successful pull image

Explore https://www.jfrog.com/confluence/display/JFROG/Artifactory+REST+API#ArtifactoryRESTAPI-ArtifactSyncDownload https://www.jfrog.com/confluence/display/JFROG/Artifactory+REST+API#ArtifactoryRESTAPI-RetrieveArtifact

These two options really do not play well with images (since its not a single file like say curl -u jose-test-user:$apikey "https://jfrog.aaw.cloud.statcan.ca/artifactory/api/download//pypi-remote-cache/00/08/c037fa5d3794729fdc7967e5a174642c8e94e1988ae2bc950515e2902ca2/Sphinx-4.2.0-py3-none-any.whl?content=none"

Cannot try and say retrieve something that doesnt exist in the cache already to pre-fetch it curl -u jose-test-user:$apikey "https://jfrog.aaw.cloud.statcan.ca/artifactory/api/download/jose-play/library/busybox/unstable-musl/manifest.json?content=none" as that will also end in failure

Adding and changing to use docker curl -u jose-test-user:$apikey "https://jfrog.aaw.cloud.statcan.ca/artifactory/api/docker/jose-play/library/busybox/latest?content=none" Again expects a file (so it will 'work' if i add list.manifest.json)

but trying to fetch something that doesnt already exist. curl -u jose-test-user:$apikey "https://jfrog.aaw.cloud.statcan.ca/artifactory/api/docker/jose-play/library/busybox/unstable-musl/manifest.json?content=none" Gives "message" : "Not Found"

Jose-Matsuda commented 2 years ago

Note there's currently an edge case where some of the notebooks are running on images that aren't from the ACR in that they don't have the leading k8s...acr.io bit

Jose-Matsuda commented 2 years ago

Might be whack should look at https://docs.docker.com/registry/spec/api/#pulling-an-image because as stated in the Docker Registry documentation Artifactory does support relevant calls of the Docker Registry API.

No it does not seem to work well either, trying the GET /v2/<name>/manifests/<reference> and any combinations of it do not work.

Also currently looking into rootless mode because docker can get pretty nasty with root permissions quick. I also wonder if there's a way to read the output of a docker pull. This is so that instead of actually pulling the image locally, once it gets to the whole Pulling from xyz then we know it has been scanned by XRAY as its getting ready to download. Because remember image If there's a vulnerability it will just get blocked.

Yeah just do (so we also catch anything unexpected) image

Jose-Matsuda commented 2 years ago

USING NERDCTL

Doing it this way image

Jose-Matsuda commented 2 years ago

Look into crane

https://github.com/google/go-containerregistry/blob/main/cmd/crane/doc/crane.md

image If i can use that config.json then its so much nicer image This could be a problem however

Trying to just fetch a manifest of something I know exists in a local repo (though this might not be an issue if not supported by Artifactory anyways, just seeing if I can get something other than pull working) image also gives the error in the artifactory service log 2022-03-18T02:49:06.767Z [jfrt ] [ERROR] [] [o.a.a.d.r.DockerResource:178 ] [ttp-nio-8081-exec-22] - Unsupported docker v2 repository request for 'artifactory'

Promising (had order of commands mixed up, found out after reading the overview and seeing the pull does remote.image into tarball. image

Jose-Matsuda commented 2 years ago

Well whaddya know image

Upon trying bitnami/node as expected you can observe image

So the change to our script in terms of the grep may be 1) For a failed (meaning vulnerable) image, look for "message" : "Forbidden" This just means the image won't try to get pulled (yay we dodged a pull) 2) For images that passed scanning successfully, look for (compressed) in cache

This may require the -v for verbose option but we will see. (Nope) image

Jose-Matsuda commented 2 years ago

Closing and tracking the ACR issue https://github.com/StatCan/daaas/issues/960