search

rust-secure-code / cargo-supply-chain

Gather author, contributor and publisher data on crates in your dependency graph.
Apache License 2.0
315 stars 18 forks source link

Add progress indication to `update` subcommand #28

Closed Shnatsel closed 3 years ago

Shnatsel commented 3 years ago

cargo supply-chain update currently shows no output, and just does its job silently. This is bad for a couple of reasons:

  1. There is no indication that a lot of data is going to be downloaded (~250Mb as of right now, and growing). This may be unexpected and undesirable for people using mobile internet connections.
  2. On slow connections it's not apparent if any progress is happening or the program just hung.

We need to provide some sort of progress indication, e.g. the amount of data downloaded so far. We cannot show a real progressbar because it would require knowing the size in advance, as well as introducing complex dependencies.

It would also be nice to show some sort of warning that a lot of data may be downloaded.

HeroicKatora commented 3 years ago

As a side note: Not all of those 250Mb are being stored, the local data is roughly 10MB right now. A lot of it is unecessary and discarded but the way that data dumps are offered (tar.gz) makes it impossible to seek to only those files that are actually relevant.

jyn514 commented 3 years ago

just use https://github.com/Nemo157/oubliette/

Shnatsel commented 3 years ago

@Nemo157 Who the Vehk puts Unicode characters into the name of their main git branch?!

Nemo157 commented 3 years ago

Someone trying to force better support for remote HEAD ecosystem wide 😉

dutt commented 3 years ago

Looking over the ureq library I think we'll have to switch http library if we want to support partial gets and some kind of progress indicators. Seems the blocking-and-get-everything call() is the only method to get a Response from a Request, unless I'm missing something.

Nemo157 commented 3 years ago

Request::call should only block until the headers are complete, you can then use Response::into_reader to stream the data out, updating a progress bar as the data is read (IIRC indicatif has support for updating the progress bar status directly from the IO source).