ipfs / kubo

An IPFS implementation in Go
https://docs.ipfs.tech/how-to/command-line-quick-start/
Other
15.83k stars 2.96k forks source link

Add progress feedback to `ipfs get` #10339

Open 2color opened 3 months ago

2color commented 3 months ago

Checklist

Description

Background

There are some situations in which ipfs get doesn't print enough progress feedback when fetching a large enough CID, e.g. 25mb.

For example, I was trying to fetch the CID QmRhzWfzRHHmMirAQrBimyuiaCRFbPprWLuyw3cBqmHUQy which is comprised of 103 blocks.

I ran ipfs get QmRhzWfzRHHmMirAQrBimyuiaCRFbPprWLuyw3cBqmHUQy which was hanging with no output (not even saying that the root block was retrieved). To make matters more confusing, I was debugging and ran ipfs routing findprovs QmRhzWfzRHHmMirAQrBimyuiaCRFbPprWLuyw3cBqmHUQy and saw my PeerID as a provider of that CID. I later discovered that only the root block QmRhzWfzRHHmMirAQrBimyuiaCRFbPprWLuyw3cBqmHUQy was retrievable and my node fetched it, however, it nothing was indicated about that in the ipfs get QmRhzWfzRHHmMirAQrBimyuiaCRFbPprWLuyw3cBqmHUQy output.

Current situation

From my testing, it seems that there's no output after the root block is fetched. It seems the progress feedback output only starts one of the first child blocks is fetched.

Another thing I noticed is that the feedback is always in size with no block information, e.g.

# Example with the Project Apollo Archives
➜  ~ ipfs get QmSnuWmxptJZdLJpKRarxBMS2Ju2oANVrgbr2xWbie9b2D
Saving file(s) to QmSnuWmxptJZdLJpKRarxBMS2Ju2oANVrgbr2xWbie9b2D
 3.99 MiB / 57.46 GiB [>-----------------------------]   0.01% 1d10h39m11s

Suggestion

When the ipfs get command retrieves the root block and it contains a links array, it could print out the CID and the number of blocks left to fetch (which can grow if fetching a directory as the dag is traversed) in addition to the number of bytes fetched, which is already printed out (even though it seems only after the first child block is fetched)

Extending this to other commands

We could also extend this feedback to other commands such as: