search

rust-secure-code / cargo-supply-chain

Gather author, contributor and publisher data on crates in your dependency graph.
Apache License 2.0
315 stars 18 forks source link

Cached data dumps #12

Closed HeroicKatora closed 3 years ago

HeroicKatora commented 3 years ago

Downloads the crates.io daily data dumps, stores the relevant information in a cache directory and uses that information to read the owner information when possible.

Shnatsel commented 3 years ago

Thanks a lot!

It's early enough in the project that I'm gonna merge now and look later :smile:

HeroicKatora commented 3 years ago

@Shnatsel Hint: You need to run cargo-supply-chain update to download the data, then the other commands will use them. It does write the publisher data for each individual release as well but that's not yet used.

Shnatsel commented 3 years ago

Do they use automatically? I think making it explicit would be nice with something like a --cached flag, otherwise it's easy to run update, forget about it and then get stale data all of a sudden.

Shnatsel commented 3 years ago

Any reason why this uses libflate and not miniz_oxide?

HeroicKatora commented 3 years ago

I did know if miniz_oxide had full gzip encapsulation support and did not want to look up the specifics of the header structure or and checksum etc.

Shnatsel commented 3 years ago

I see. Nope, you have to use flate2 for that :crying_cat_face: