Incomplete Metadata because of combination with downloader

owntone / owntone-server

Linux/FreeBSD DAAP (iTunes) and MPD audio server with support for AirPlay 1 and 2 speakers (multiroom), Apple Remote (and compatibles), Chromecast, Spotify and internet radio.

https://owntone.github.io/owntone-server

GNU General Public License v2.0

2.03k stars 234 forks source link

Incomplete Metadata because of combination with downloader #1782

Closed Bockiii closed 3 weeks ago

Bockiii commented 1 month ago

Greetings,

I'm using the lsio/daapd docker image, but since they are building from source, I assume this happens in other cases as well.

My setup: Unraid as host /Audiobooks as a folder on Host, mountable by docker containers "Deemix" running as docker container with /Audiobooks as a /Downloads folder (so it immediately writes into that folder) owntone running as a docker container with /Audiobooks mapped to /Audiobooks within the container.

Steps to reproduce:

Download an album in deemix (including cover art)
Look at owntone

Expected behavior:

Owntone will show the albums when they come in with cover art and correct cover art etc.

Actual behavior:

Owntone will show an "unknown artists" entry with the mp3 names as values in the list. No ID3 info or cover data available

I attached the debug log. The settings for this log were "loglevel = debug" and "filescan_disable = true" so that it doesnt scan the 100k+ files in the folder with debug level logging. owntone.log

Bockiii commented 1 month ago

whatdoineed2do commented 1 month ago

based on the comment in the PR, it sounds like you're saying the inotfiy event for "update" to file in place for the metadata does not trigger the server's rescan of file.

For me, this "unseen" activity happens on remote NFS filesystems.

You may want to check what the inotify events are happening and to check what your download client is doing when its finalising the downloaded file with metadata

Bockiii commented 1 month ago

based on the comment in the PR, it sounds like you're saying the inotfiy event for "update" to file in place for the metadata does not trigger the server's rescan of file.

For me, this "unseen" activity happens on remote NFS filesystems.

You may want to check what the inotify events are happening and to check what your download client is doing when its finalising the downloaded file with metadata

I have no idea how I would check the inotify events apart from looking at the log file. It doesnt seem like it registers the update, but I can't say why. The downloader log just says "done downloading" :)

In this case its not a remote nfs, its a local folder mounted into the docker container

ejurgensen commented 1 month ago

You don't need to check this. If inotify wasn't working you wouldn't see the file at all. I will look into the issue you made, but I'm away right now.

Bockiii commented 1 month ago

i dont know if inotify has a cooldown of some sorts (as I have no idea how it works at all), but maybe the downloader fires off too many quick edits at the end to save the file and then adapt the id3 tags or so... no idea

I also noticed that, if I download a whole lot, its possible that it picks up "some" files with correct id3 tags, which also leads to incomplete albums showing with only title 3, 12 and 32 or something like that. So it could be a timing/cooldown issue.

Bockiii commented 4 weeks ago

You don't need to check this. If inotify wasn't working you wouldn't see the file at all. I will look into the issue you made, but I'm away right now.

any progress? Are you back already? :)

ejurgensen commented 3 weeks ago

Yes, back now and had a look at the log now. I appreciate that you cut some of the irrelevant stuff, but it would have been even better if you had made a simpler test. Here is it seems 39 tracks + a playlist are being added, so the log is full of noise.

I think I can see a possible cause, but needs a bit more investigation.

ejurgensen commented 3 weeks ago

The issue should be fixed now. The cause was that the file wouldn't be rescanned if it was modified within the same second of a previous modification. Looks like this was an old bug, going back at least 7 years. So thanks for highlighting it so it can finally be cleaned up.

Note the fix is only in master right now. I don't know if there are Docker images matching that, otherwise I expect you will have to wait fot the next release.

Bockiii commented 3 weeks ago

Awesome thank you!

I have made a script for the mean time (just a find command in combination with touch) to fix my stuff in the mean time without having to rescan the whole data as a workaround, but I'm happy that this will be fixes with the next release!

hacketiwack commented 3 weeks ago

I don't know if there are Docker images matching that, otherwise I expect you will have to wait for the next release.

The version owntone/owntone:staging on DockerHub includes this fix.

ejurgensen commented 3 weeks ago

Excellent @hacketiwack. Then I hope you test with that and confirm if the fix works, @Bockiii.