M4SS-Code / cargo-goggles

Verify that registry crates in your Cargo.lock are reproducible from the git repository
https://crates.io/crates/cargo-goggles
Apache License 2.0
36 stars 2 forks source link

Feature request: Better explainatons of the problems found #4

Open VorpalBlade opened 7 months ago

VorpalBlade commented 7 months ago

Consider some lines from my first run:

Package addr2line has mismatching file hashes for addr2line-0.21.0/Cargo.lock

Maybe show a diff? Or provide some other easy way to inspect the actual differences. Or is this just due to the library not having a checked in Cargo.lock?

Commit between crates.io tarball and git tag doesn't match for ahash v0.8.11

Aaah! I definitely want more details on that one! (Will be investigating that one manually)

Couldn't publish the package in https://github.com/tkaitchuck/constrandom.git repo status=exit status: 101

Seen this one several times. It would make sense to show more details here.

Couldn't checkout the commit in https://github.com/briansmith/untrusted.git repo status=exit status: 128

Similar to the above, I want more info.

Package still somehow doesn't exist ahash

As an error message, this doesn't stand out enough from the log lines (whatever it means).

VorpalBlade commented 7 months ago

Perhaps there could be a flag --investigate package-name=1.2.3 flag or similar that checked out both in directories below the current working directory, and did a soft publish and unpacked that as well. Then it would make it easy for me to manually investigate what is going on.

VorpalBlade commented 7 months ago
Found NO tag match with package compact_str
Found NO tag match with package const-random
Found NO tag match with package const-random-macro
Found NO tag match with package generic-array
Found NO tag match with package heck
Found NO tag match with package heck
Found NO tag match with package hermit-abi
Found NO tag match with package ident_case

This seems a good reason to also check whatever commit ID that cargo identified when publishing (if it exists). I would argue checking both and making a summary table at the end.

paolobarbolini commented 7 months ago

Thank you very much for all of the feedback. I've spent most of the time scanning crates.io instead of improving the code, but I definitely want this tool to become useful and easy to use in the medium to long term.

The NO tag match error is for crates that don't seem to have tagged the release in question in the git repository. I have already opened issues for a few of them. A lot of maintainers seem to forget to tag or to push tags for new releases. Many crates surprisingly don't use tags at all, which combined with the lack of .cargo_vcs_info.json, or publishing from unknown branches, makes the process of reproducing the release much harder.

paolobarbolini commented 7 months ago

I've just seen https://github.com/briansmith/untrusted/issues/69#issuecomment-2029733947. Not to downplay issues, but given that many maintainers commit from release branches, it seems a bit alarmist (though arguable, given the scope of the crate) to post a comment like this. I wouldn't want to alienate the community like RustSec did with some maintainers that didn't agree with unmaintained crate advisories (probably because of the backslash from the downstream users)

VorpalBlade commented 7 months ago

You are right, I changed from deeply problematic to somewhat problematic.