search

paritytech / parity-publish

A tool to manage publishing Parity's crates
Apache License 2.0
5 stars 1 forks source link

Use `crates.io` index repo #11

Closed ggwpez closed 11 months ago

ggwpez commented 1 year ago

Command status is running rather slow on the monorepo and seems to do one request per crate?
Dont know if this was thought about before, but the crates.io index is on github.
So you can get all versions of all crates at once by downloading that repo.

Morganamilo commented 1 year ago

Seems reasonable. The index is actually smaller than expected, I thought it would be dozens of GB.

ggwpez commented 1 year ago

You can also do a --depth 1 clone, or rather just access the latest master zip directly without any .git overhead:
https://github.com/rust-lang/crates.io-index/archive/refs/heads/master.zip

Looks like it is about 107 MB currently.

Morganamilo commented 1 year ago

We'll probably need to pull the newest index on each invocation of this tool so going through git is probably the best way.

Morganamilo commented 11 months ago

we use the cargo registry where we can now which is very fast. We still have to use the http api for ownership as that.s a crates.io thing and not a registry thing. That's now dones in parallel so that's not too slow anymore either.