advplyr / audiobookshelf

Self-hosted audiobook and podcast server
https://audiobookshelf.org
GNU General Public License v3.0
6.36k stars 448 forks source link

[Enhancement]: Switch from inotify to alternative for CIFS shares #3469

Open KaiStarkk opened 3 days ago

KaiStarkk commented 3 days ago

Type of Enhancement

Server Backend

Describe the Feature/Enhancement

CIFS mounts apparently aren't able to detect SMB2+ change_notify events for Linux clients. Incidentally, this is not the case for Windows clients using ReadDirectoryChangesW.

For this reason, it would be nice to have a polling alternative for Linux (or docker) clients such as:

I could be wrong, but it appears that the built in scanner is more resource intensive and slow than above tools which would be based on stat. If I am wrong there this FR is redundant.

Why would this be helpful?

This would prevent having to manually scan libraries after books are added.

Future Implementation (Screenshot)

N/A

Audiobookshelf Server Version

v2.13.3

Current Implementation (Screenshot)

No response

advplyr commented 3 days ago

Abs uses watcher to watch for file changes.

I'm not sure we want to add in polling due to how large some users libraries are. Keeping the record of and polling 100k+ files I can imagine being very expensive, especially over NFS. I would be curious how other media servers handle this.

We wouldn't use the libraries you shared because Abs is built with nodejs.

mikiher commented 3 days ago

For the record, here's a discussion around this topic from an older bug.

KaiStarkk commented 2 days ago

Thanks @mikiher . I like the groundwork you've laid for a Windows watcher, I'll eventually do something similar for Linux hosts (which should also cover TrueNAS Scale) and maybe some of the BSD NAS OSs.

@advplyr One thing that would help a lot would be an API endpoint or payload in the scan endpoint to specify a directory to be scanned, rather than the whole library. That would make things much more responsive. I assume this is what the server already does internally when the watcher detects a change? Happy to look into it

mikiher commented 16 hours ago

When I wrote AbsWatcher, we opened an endpoint, /watcher/update for doing something very similar to what you describe. AbsWatcher is using it (it is currently undocumented, without good reason). It is implemented here

I thought a little bit in the past on how to generalize AbsWatcher to run on any file-serving system:

AbsWatcher has the advantage that Audiobookshelf Windows Docker and the files it serves are on the same system, so it has access to the docker command, which it uses to find the mapping between docker volumes and their respective windows folders. But if you're able to provide that mapping yourself (either manually or programmatically), the rest of the code (watching directories, notifying ABS through the API when changes happen) remains the same.