search

mutagen-io / mutagen

Fast file synchronization and network forwarding for remote development
https://mutagen.io
Other
3.45k stars 153 forks source link

Display detailed scan progress (including the current path) in monitoring #143

Open FrankHarland opened 5 years ago

FrankHarland commented 5 years ago

I'm spending a lot of time looking at "scanning files" not knowing what files are being scanned at that moment, maybe it's a folder that can be excluded?

Would be nice if we could see what is being scanned. I've scanned through the docs but there does not seem to be an option for this.

xenoscopic commented 4 years ago

Thanks for reporting this, sorry for the really slow reply. I absolutely agree with everything you've said. There are essentially three issues blocking a fix here.

The first is that the monitor command's printing code is fairly rudimentary, and the trickery that it uses to update line contents limits both the line length and the number of lines that can be dynamically updated. If either of these two limitations could be lifted, it would be easier to print this information. I'm not sure if this would require updating to a full curses-based monitoring implementation or if there's a simpler solution. I plan to have a look at what Docker uses for its dynamic text displays (e.g. when it shows multiple layers being pulled concurrently), because it might be a simple and easy fit for Mutagen.

The second issue is finding a way for the scanner itself to provide state callbacks without significantly slowing down the scan. This may actually be a non-issue, but it will require careful benchmarking.

The third issue is figuring out how to handle state updates from remote endpoints. The current synchronization protocol doesn't really provide a mechanism for feeding this information back in a streaming fashion. This can certainly be implemented, but it will require some sort of rate limiting and/or downsampling to avoid hurting bandwidth and latency.

In the meantime, the best suggestion I have for debugging slow scans would be strategic use of the du command or a tool like GrandPerspective to identify large directories. Things like node_modules are the usual candidates. VCS directories can also be quite large (and should be ignored).

Thanks again for the feedback! I'll keep this issue updated with progress.

hjanuschka commented 3 years ago

not a solution - but i had same issue. and going to the beta, where agent runs - find it's PID

and then ls -l /proc/5249/fd

atleast shows the last couple of files:

Bildschirmfoto 2021-05-20 um 18 43 15

not perfect, but atleast i know it is still "working"