cboxdoerfer / fsearch

A fast file search utility for Unix-like systems based on GTK3
GNU General Public License v2.0
3.33k stars 209 forks source link

Read changes on Samba mounts #401

Open danielkrajnik opened 2 years ago

danielkrajnik commented 2 years ago

NTFS mounts provide "USN Journal" listing recent changes, which voidtool's "Everything" uses to keep database up to date as soon as changes are made on a network share.

It's mentioned for example here: https://www.voidtools.com/forum/viewtopic.php?f=5&t=11919

Unfortunately, plocate/mlocate hasn't yet implemented a feature to updatedb only on recent changes held in USN Journal. It seems like a useful feature, which I realize has been requested already (#26 ) and would be more effcient than scanning entire directory at least for Samba shares. It would be amazing if fsearch could have it.

cboxdoerfer commented 2 years ago

Hi, I don't think it's possible to access the USN journal through Samba. Are you sure Everything can do that?

I guess one workaround would be to run a separate FSearch instance on the Samba server, which then monitors the USN journal and sends all changes to all FSearch clients over the network with a custom protocol, but that's quite a lot of work.

danielkrajnik commented 2 years ago

I think that Everything uses "change notification" function of SMB servers which should be also available from samba shares:

https://stackoverflow.com/questions/66712960/can-smb2-change-notify-be-used-to-keep-an-accurate-snapshot-of-a-remote-director/66816471#66816471

https://www.linuxtopia.org/online_books/network_administration_guides/using_samba_book/ch08_05_01.html#:~:text=The%20change%20notify%20timeout%20global,server%20will%20notify%20the%20client.

I've seen Everything make updates in real time on windows shares as well, although it does eat up a lot of bandwidth, so if possible this might be rather an "opt-in"? It would be definitely useful though for some users.

For reference, one user mentions here "SMB CHANGE NOTIFY" events sent from Everything: https://www.voidtools.com/forum/viewtopic.php?t=7095

Hope that makes sense, perhaps this issue should have been called "...change_notify..." rather than "...USN Journal..."

cboxdoerfer commented 2 years ago

@danielkrajnik Thanks for those links and information. I think that's something worth looking into. Unfortunately this seems to be a little bit more difficult on Linux compared to Windows or macOS: https://lwn.net/Articles/896055/

But I will have a closer look at that once the monitoring for changes of native filesystems is implemented.

UkeHa commented 1 year ago

@cboxdoerfer would it be possible to implement this feature with nfs?

cboxdoerfer commented 1 year ago

@UkeHa No, I don't think so. Last time I checked (a few months ago), the Linux kernel still doesn't implement any filesystem notifications for NFS.