windirstat / windirstat

WinDirStat is a disk usage statistics viewer and cleanup tool for various versions of Microsoft Windows.
https://windirstat.net
GNU General Public License v2.0
524 stars 50 forks source link

New Time calculation inaccurate #14

Open AartBluestoke opened 8 months ago

AartBluestoke commented 8 months ago

runtime time is only accurately reported for 1 thread. I've seen it both faster and slower (here is a slower - clearly you can't do a refresh in only 1 second)

here is a run with 6 seconds wall clock, 49 seconds runtime reported image

here is a refresh of an existing run that reports completion in only 1 second (it took the normal time visually) image

NoMoreFood commented 8 months ago

Yup, this is on the to-do list. It's a consequence of multi-threading. Do you find the total time spent helpful? I'm questioning whether it might be just right thing to remove it altogether.

Curious... on your drives... if you run it, close it, and run it again. How long does it take to run?

AartBluestoke commented 8 months ago

10 seconds with 4 threads, down to 5 seconds with 16 threads - the 6 seconds above was with ~8 threads selected. my drives are 2x https://www.crucial.com/ssd/p5-plus/ct2000p5pssd8 (2TB) cpu is 12900H i'm on windows 11.

AartBluestoke commented 8 months ago

as another time point for execution speed https://locate32.cogit.net/ (https://github.com/Locate32/Locate32) appears to be able to index my system single-threaded in ~20 seconds.

assarbad commented 7 months ago

I find the wall clock time most useful. Thread times can be intriguing for the tech-savvy, but do they really play a role?

@AartBluestoke did you also empty the lookaside cache (or reboot) between your test runs? Otherwise any subsequent tests will always be faster, given a machine that isn't under a lot of cache stress. The RAMMap utility from Sysinternals/Microsoft offers the option to empty said caches.

NoMoreFood commented 7 months ago

Yeah, I should have been clearer -- I don't actually think the cumulative thread time is useful -- it was just a consequence of the multithreading change. Algorithmically, I'll have to think hard about we can accurately track wall clock time for each branch at each level when there could be multiple threads beneath it starting and stopping at different times while not harming performance or eating up extra memory.

AartBluestoke commented 7 months ago

@AartBluestoke did you also empty the lookaside cache (or reboot) between your test runs?

  • no - i'll do another run now ( i recently rebooted) - it took 10 to 11 seconds (my hand-timing was poor, and there is a pause between the completion of the sweep and the display of the treemap (i try to aim at the end of the treemap display)
  • image

AartBluestoke commented 7 months ago

an immediate re-execution was about half the time (currently set to 10 threads) image

NoMoreFood commented 7 months ago

@AartBluestoke Can you try the latest commits?

AartBluestoke commented 7 months ago

new behaver - time is working, but scan is broken - i've installed google drive, and this appears to have caused windirstat to hang - it just spins with this screen, gradually increasing ram ... image

AartBluestoke commented 7 months ago

edit: not google drive - i closed it and saw the same behavior. image

NoMoreFood commented 7 months ago

@AartBluestoke Thanks I'll take a look. Does it occur if you target a single folder?

NoMoreFood commented 7 months ago

Actually I think I see the problem. I committed a small change that I think should address the behavior. I had rearranged a few things for the timing and injected a bug along the way.