P2P code is not Byzantine fault tolerant

[mwleeds]

In ostree's P2P code paths, repositories provide unsigned summary files along with signed per-repo and per-commit metadata that corroborates the important information in the summary. However, the signed metadata isn't checked at the same time as the summary file, and decisions are made based on the data in the summary file, so a maliciously crafted summary could DoS a peer on the same network (attacks using Internet remotes are more difficult, see here). Specifically, it's find_remotes_cb that reads the unsigned summary files. So if a peer claims to have the latest versions of every ref, find_remotes_cb would prioritize it and ostree_repo_pull_from_remotes_async would pull from it and fail, never getting any updates (because pull_from_remotes_async doesn't pull refs mapped to NULL checksums, which are refs that seem outdated or missing).

It would be fun to try a proof of concept of this, but I haven't yet.

Some options:

1. Make it so that pull_from_remotes_async pulls refs from remotes that don't claim to have the latest commit, only when the pull from the remote claiming to provide the latest commit failed.
2. Pull and check the signed commit metadata to verify the information going into the OstreeRepoFinderResult data structure. Alternatively, move some information mapping refs to commits into ostree-metadata and use that. This wouldn't prevent a peer from claiming to provide the latest real commit, only from claiming to provide a fake future commit. But I think that would be enough to prevent DoS.
3. Don't do anything, because the attack requires being on the same network as the victim, which usually implies some amount of trust.
4. Use a blockchain™ (jk)

Moved this into its own issue from https://github.com/ostreedev/ostree/pull/1518#issuecomment-377349842

[pwithnall]

• Make it so that pull_from_remotes_async pulls refs from remotes that don't claim to have the latest commit, only when the pull from the remote claiming to provide the latest commit failed.

This seems reasonable, and was what I intended when writing the initial implementation of pull_from_remotes_async() — that it could be made more intelligent in future wrt pulling from multiple remotes in parallel, falling back to older versions if the newer ones aren’t available at all, etc.

Note that we don’t need to consider an attack to motivate falling back to pulling older commits (which are still newer than what the local machine has) from remotes: the same is necessary if the peer with the latest commits goes offline between finding it and pulling from it.

• Pull and check the signed commit metadata to verify the information going into the OstreeRepoFinderResult data structure. Alternatively, move some information mapping refs to commits into ostree-metadata and use that. This wouldn't prevent a peer from claiming to provide the latest real commit, only from claiming to provide a fake future commit. But I think that would be enough to prevent DoS.

I avoided that in the first implementation because I think it would result in too many requests and too much data being pulled. For example, if you’re updating 10 apps on a LAN with machines which advertise 10 different versions of those apps (due to not being up to date, or whatever), that’s 100 requests for commit metadata, and 100 commit metadata files which you need to store locally for a while until they get pruned later on.

• Don't do anything, because the attack requires being on the same network as the victim, which usually implies some amount of trust.

That’s OK for the initial implementation (it’s what we currently do), but I think it’s not a long-term solution. While we can trust local networks in some contexts (a well managed company or school, for example), we might not be able to trust them in every context (for example, consider a home network where some IoT device has been compromised) — so I wouldn’t want to put all our eggs in one security basket long term.

• Use a blockchain™ (jk)

srsly.