edgelesssys / contrast

Deploy and manage confidential containers on Kubernetes
https://docs.edgeless.systems/contrast
GNU Affero General Public License v3.0
160 stars 6 forks source link

Minimise TCB on during verification by breaking out the verify subcommand #598

Open blenessy opened 1 week ago

blenessy commented 1 week ago

contrast v0.7.0 is very big ~60 MB in size with DWARF and symbols removed.

I've tested breaking out the verify subcommand from the contrast CLI into its own binary (called verify). The size of verify is 9.7MB. I analysed the contents of this binary with GSA. I attached the html report so you can see for yourselves. Spoiler: bigger code chunk is related to the GRPC protocol.

Do you guys think this is a good idea to do this (breaking out verify). Would you accept a PR with separated verify binary ?

(FWIW. I would put in more effort to further minimise TCB of verify after the separation - I'm hoping to bring the size down to 4-5 MB).

verify.html.gz

burgerdev commented 1 week ago

Hi @blenessy,

Thanks for the suggestion, I'm pretty happy that you're taking Contrast for a spin and considering contributions!

Regarding Trusted Computing Base

I fear that a reduction in binary size does not necessarily imply a reduction in TCB. Even if you compile a small verify binary and use it to get the Coordinator attestation doc and manifest, you still need to verify them. Things that come to mind:

Now you might ask yourself why the policies obtained by `verify` are not sufficient for verification. This is mostly due to how workload policies in Kata operate (I wrote a paragraph about that [here](https://github.com/edgelesssys/contrast/blob/v0.7.0/dev-docs/coco/policy.md#storage-rules)): given a bunch of yaml it is easy to verify that `generate` produces the same result as retrieved by `verify`, but working backwards from policies to k8s resource definitions is infeasible, if only because of the precalculated dm-verity hashes for the OCI layers. This may change in the future, depending on the image handling strategy decided by upstream CoCo, but we're stuck with it for now.

At the end of the day, the TCB is the transitive closure of all Contrast components and their dependencies. If the goal is to reduce the TCB, we should imho start by reducing dependencies overall.

Regarding a Standalone Verify

That being said, I can imagine situations where a smaller verify binary would be useful, even if the total TCB is unaffected. Built with the correct reference values and somehow equipped with a manifest through a side-channel, this could make verification feasible even on very constrained systems.

On the other hand, we still want to support the verify subcommand in the main binary, and we're not really eager to maintain this functionality in two binaries. However, if this is useful to you and you see a low effort way to add and maintain that second binary, I'd be open to adding it to a contrib folder, for example.

Regarding Binary Size Overall

There are three main contributors as far as I'm aware of (see table below):

These are all used by contrast generate , so we can't avoid packaging these with the CLI.

```plain $ gsa ./contrast +-----------------------------------------------------------------------------+ | contrast | +---------+----------------------------------------------+--------+-----------+ | PERCENT | NAME | SIZE | TYPE | +---------+----------------------------------------------+--------+-----------+ | 38.59% | .noptrdata | 23 MB | section | | 16.30% | k8s.io/api | 9.8 MB | vendor | | 12.47% | .gopclntab | 7.5 MB | section | | 10.55% | .rodata | 6.4 MB | section | | 2.80% | k8s.io/client-go | 1.7 MB | vendor | | 1.82% | github.com/google/gnostic-models | 1.1 MB | vendor | | 1.62% | google.golang.org/protobuf | 980 kB | vendor | | 1.48% | k8s.io/apimachinery | 893 kB | vendor | | 1.46% | crypto | 879 kB | std | | 1.27% | net | 764 kB | std | | 0.98% | google.golang.org/grpc | 589 kB | vendor | [...] ```

Cheers, Markus

blenessy commented 1 week ago

Thanks for the quick and very insightful response @burgerdev !

As you suspected, I was under the assumption contrast verify + the evidence downloaded (by contrast verify) to the ./verify directory is relatively easy to process. But it does sound like it is not as straight forwards as I anticipated :).

I will definitely start by exploring your concerns more in depth before moving forward with this.