search

awslabs / tough

Rust libraries and tools for using and generating TUF repositories
191 stars 43 forks source link

client: hashes and length should be optional in (timestamp, snapshot) METAFILEs #771

Open jku opened 1 month ago

jku commented 1 month ago

I was sure this issue already existed but now I cannot find it...

tough client does not seem to support METAFILEs without hashes or length within timestamp and snapshot metadata.

The specification is clear in this case: these are optional fields that can be omitted:

LENGTH An integer length in bytes of the metadata file at METAPATH. It is OPTIONAL and can be omitted to reduce the snapshot metadata file size. In that case the client MUST use a custom download limit for the listed metadata.

HASHES A dictionary that specifies one or more hashes of the metadata file at METAPATH, with the cryptographic hash function as key and the value as HASH, the hexdigest of the cryptographic function computed on the metadata file at METAPATH. For example: { "sha256": HASH, ... }. HASHES is OPTIONAL and can be omitted to reduce the snapshot metadata file size. In that case the repository MUST guarantee that VERSION alone unambiguously identifies the metadata at METAPATH.

I am currently not including these fields in what is likely to be sigstore TUF repository in future. This lead to https://github.com/sigstore/sigstore-rs/issues/369

flavio commented 1 week ago

I'm willing to look into that and eventually build a patch for that. I tried to reproduce the issue you mentioned using the sigstore-staging TUF repository, but this is currently malformed (as you pointed out inside of the sigstore-rs issue).

I've looked into tough and I think METAPATH is handled by SnapshotMeta. This one has both length and hashes optional. Everything seems to be fine :thinking:

I'll look more into that once I'm able to reproduce the issue

jku commented 4 days ago

huh! I can make a test repo for this tomorrow -- it's of course possible I'm somehow mistaken

jku commented 4 days ago

Actually there is a test repo:

$ curl -o root.json https://jku.github.io/tuf-demo/metadata/3.root.json
$ tuftool download --root root.json -m https://jku.github.io/tuf-demo/metadata/ -t https://jku.github.io/tuf-demo/targets -n file1.txt out/
Failed to load repository: Failed to parse timestamp metadata: missing field `length` at line 14 column 4

So I suppose the issue might be only with timestamps METAFILE?

flavio commented 4 days ago

@jku thanks for the reproducer. The problem was indeed with how timestamp.json was parsed. I've created https://github.com/awslabs/tough/pull/778 to address that. With these changes I can download your test TUF repository.

I hope someone from the TUF maintainers will take a look at that PR

jpculp commented 3 days ago

Thank you @jku for raising this issue and thank you @flavio for addressing it! Sorry we didn't catch this sooner.