No need to diff files

[hauleth]

Git internally is just a Merkle-tree so there is no need for diff as you can just compute Git-compatible tree and compare it with the one stored in repo. In theory it can end with differences (as SHA-1 is broken), but it would allow to speedup first check and reject obvious invalid repos.

[nietaki]

Thanks for the suggestion!

I like this idea, especially since it would remove hoplon's dependency on diff. I had a quick look at how Merkle-trees are used in Git and it looks interesting and viable. There's a handful of things holding me back from jumping on it though.

• I like the idea of hoplon being simple and having as few dependencies as possible. To go the Merkle-tree route we'd either need to take a dependency for it or write all the relevant code inside hoplon itself.
• From my experiments with hoplon so far it doesn't seem like the time it takes to diff is the deciding factor in the whole operation. For example, for Ecto, which is one of the heavier dependencies you can take on, it's basically instant on my machine. The time it takes to make all the network calls is much more significant. I'm also curious how the overhead from calculating the hashes would compare to the overhead of using diff.
• as you mentioned SHA-1 is broken, so it would potentially give people one more attack vector compared to naively comparing the files. Keeping it simple might actually be somewhat of a benefit here.

I'll have a closer look at this idea in the future, but as I see it now, I don't think the benefits are strong enough. I'm happy to be proved wrong though :)

[hauleth]

@nietaki what I am suggesting is to have two step verification:

1. Compute Merkle-Tree of files in repository and in dependency tree. While it requires some code, it isn't that much as Erlang has built-in SHA-1 in crypto module.
2. If everything is valid then run full comparison.

Alternatively you can compute 2 Merkle-trees using two different functions and this would greatly reduce attack surface (ex. SHA-1 and compare it with Git and then SHA-256 both in Git and deps to be sure, SHA-256 isn't currently broken and even if it was I do not thin that it would be feasible to find one collision in both functions at the same time).