search

syncthing / syncthing

Open Source Continuous File Synchronization
https://syncthing.net/
Mozilla Public License 2.0
65.55k stars 4.31k forks source link

Improve scan speeds using MFT #8795

Closed lukacat10 closed 1 year ago

lukacat10 commented 1 year ago

Currently, scan times are quite slow (on HDD specifically in my case). I witnessed similar issue being solved when I was informed on the existence of the ultra fast wiztree graphical disk usage analyzer compared to the old, trusty-rusty windirstat.

The use of Master File Table (MFT) is the reason for such scan speeds.

The MFT contains metadata on all files stored on the volume in one single structure.

MFT exists on NTFS. ext4, from I have seen, has a metadata, which is quite a bit different and might not have such an edge in terms of speed.

I believe its worth it even if its only for all the windows users.

I looked through the code, and I haven't seen anything related to MFT in the project on the main branch, but I might miss something.

calmh commented 1 year ago

It's not clear to me what this means in practice.

rasa commented 1 year ago

Nor is there a stable MFT library in go, that I can see. They are all pre-beta.