datastore: `ipfs repo stat` seems to hang with large repos on Windows

mateon1 commented 7 years ago

Version information:

0.4.5-dev, 56058c1 with HAMT sharding merged in (unrelated)

Type: bug

Priority: P2

Description:

On a repo with 312695 objects, on a HDD formatted with NTFS, ipfs repo stat seems to take forever to finish. I assume it's doing something inefficient when checking for filesizes, as ipfs refs local finishes in 18 seconds.

C:\Users\matin>sh -c "time ipfs repo stat"

Error: request canceled

real    12m41.638s
user    0m0.000s
sys     0m0.031s

C:\Users\matin>sh -c "time ipfs refs local | wc"
 312695  312695 14728215

real    0m18.022s
user    0m0.375s
sys     0m0.952s

(on a side note, refs local also seems slightly slow, but that might just be an IO bound)

hsanjuan commented 4 years ago

I think this was fixed at some point by including repo-size estimations in flatfs. Please try upgrading.

trendsetter37 commented 1 year ago

I am getting this behavior as well on ipfs version 0.21.0. I currently have terabytes in a local private cluster so while it's technically usable i'm interesting on how this is suppose to scale

Update: I cancelled this after about 30 minutes

  % sh -c "time ipfs repo stat -H"                                                                                                                                    !10237
^C
Error: context canceled

real    29m0.527s
user    0m0.287s
sys     0m0.103s

ipfs / kubo