search

projectatomic / atomic

Atomic Run Tool for installing/running/managing container images.
Other
526 stars 139 forks source link

pulling with --storage ostree doesn't enforce signatures #829

Open aweiteka opened 7 years ago

aweiteka commented 7 years ago

When running atomic pull with ostree storage backend signature policy is not enforced because the skopeo layers command doesn't enforce signatures (see https://github.com/projectatomic/skopeo/issues/287). Is there a reason we're not using skopeo copy here? https://github.com/projectatomic/atomic/blob/master/Atomic/syscontainers.py#L1172

/cc @giuseppe

rhatdan commented 7 years ago

I don't think skopeo supports an ostree backend? @runcom

runcom commented 7 years ago

@rhatdan it doesn't, it relies on a-now-deprecated command called "skopeo layers" to import individual layer to an ostree repo on a system.

rhatdan commented 7 years ago

Right so to get skopeo copy to work we would need to have a destination like ostree:/var/lib/containers and put the code to install the layers into skopeo correct?

runcom commented 7 years ago

Yeah, that would be nice, @giuseppe what do you think? I remember some conversion with @giuseppe and @mtrmac somewhere but I couldn't find the link anywhere right now.

mtrmac commented 7 years ago

A low-effort fix might be to get https://github.com/containers/image/pull/159 + a corresponding skopeo enhancement, and use skopeo copy --layer-subset instead of skopeo layers. That would allow us to preserve the existing Python code, at the cost of having to be really careful so that we always do verification, and only pull exactly one image (I suppose, first pull the manifest + no layers, and then use digest references, using a digest computed from that manifest, for all subsequent operations).

Alternatively we could build an ostree ImageTransport directly into containers/image in Go (and drop the Python code from atomic?).

rhatdan commented 7 years ago

I would prefer solution #2 we could build an ostree ImageTransport directly into containers/image in Go (and drop the Python code from atomic?).

runcom commented 7 years ago

The key functionality we need to port from python to go in containers/image is the ability to check for already pulled layers to do incremental pulls on image update (@giuseppe can tell us more about it I guess). But with the changes in c/image such as HasBlob we should be able to maintain that compatibility.

cgwalters commented 7 years ago

I'd at some point like to be able to pull OCI from ostree, or at least rpm-ostree. See discussion in https://github.com/ostreedev/ostree/pull/619

rhatdan commented 7 years ago

That would be very cool

mtrmac commented 7 years ago

containers/image does have a (manifest) format conversion machinery which could be useful for this, even if images did not originate as OCI images (or the other way around).

giuseppe commented 7 years ago

From my understanding of https://github.com/projectatomic/skopeo/pull/242, skopeo copy doesn't allow us to specify what layers to download yet. That is a blocker for atomic pull --storage ostree, as we want to fetch only the layers that are not already in the repository (shouldn't this be done for Docker as well?).

atomic pull --storage ostree fetches the manifest, lookup the layers of the image and then request to skopeo layers only those that are not already in the OSTree repository.

aweiteka commented 7 years ago

It appears skopeo copy pulls down manifest schema v2 (if available) and skopeo layers pulls down manifest schema v1.

mtrmac commented 7 years ago

It appears skopeo copy pulls down manifest schema v2 (if available) and skopeo layers pulls down manifest schema v1.

True; and actually unnecessary nowadays AFAICT, we can parse the v2 data as well; it seems to just be a leftover implementation artifact.

mtrmac commented 7 years ago

Really the question here is whether we should decide to port all of this to Go, or whether to keep the current code and only to implement https://github.com/containers/image/pull/159 and make it available in skopeo. (Though we now have a PR to implement something like that anyway, https://github.com/containers/image/pull/218, so that may happen whichever way we decide to take.)

rhatdan commented 7 years ago

I think we implement it in Go and drop it from atomic. It would be a good example of pulling and eventually pushing images using skopeo using different storage backends.

giuseppe commented 7 years ago

in the meanwhile, how much work would be to filter the layers in Skopeo? I guess the work to adapt in atomic to adapt it won't be much, and we can start using it while we wait for a better solution either in Skopeo or in OSTree (which would be shared with flatpak)