aagon
commented
1 year ago Update : the missing trailing slash in the aforementioned PROPFIND request is confirmed to be the cause of the very long update time.
A simple server-side workaround is to rewrite the request before serving it. For instance, in lighttpd :
webdav.activate = "enable"
webdav.is-readonly = "enable"
dir-listing.activate = "enable"
url.rewrite-if-not-file = ( "^(.*[^/])$" => "$1/" )When the server is setup like this, the malformed PROPFIND requests are corrected and dealt with correctly almost immediately, resulting in very reduced update times.
Note that it is very important that the adding of the trailing slash is made only on collections, NOT on regular files, using conditional rules like url.rewrite-if-not-file (or the nginx or apache equivalent). This is because mpd interprets a trailing slash as evidence a file is a collection and needs to be treated as such (by issuing a new PROPFIND request of depth 1 on it, to find its contents). As a side remark, maybe it would be better to rather use either the getcontenttype or the resourcetype property, since mpd requests it anyway and the WEBDAV standards stipulates they MUST be provided. Generally speaking, a trailing slash on a URL or href does not mean at all that a file is a directory.
Naturally, this is not particularly satisfying, as correction should be guaranteed on the client side as well. The offending line has been found, a PR will be added shortly, discussion will move there.
MaxKellermann
Bug report
Describe the bug
When the music files are stored on a distant webdav server, for example :
and the database is local :
Triggering a database update after no change whatsoever to the files causes the (unreasonably slow, even when taking into account that the files are remote) removal of all the music files, and their subsequent re-adding (at reasonable speed considering the files are distant) immediately after.
Expected Behavior
We would expect the database update to cause no removals and no additions, since nothing has changed on the remote filesystem.
Actual Behavior
We have the complete emptying of the database, followed by its immediate refilling (the end result is the same as what we had before the database update).
Version
Configuration
The website
musique.ricorambo.suis a real one, it can be used for reproducing. I advise to reduce to a specific subdirectory, as I've done here, to gain time.Log
The relevant part of the mpd log, starting with a clean database :
The relevant webdav logs :
Personal enquiries and hypotheses
The most interesting line in my opinion is this one :
I may be wrong, but
PROPFINDis never supposed to return an empty response. When I tried the same request with curl interactively, curl just waited for the end of the response that never came (I did not have the patience to wait until the likely timeout).The empty response may have something to do with the missing slash at the end of the URL : in fact,
1998%20-%2030-%20Very%20Best%20Ofis a directory, not a regular file.Another clue is that this problem is not reproduced if I set
music_directoryto a directory that does not contain any directories itself, like"http://musique.ricorambo.su/Deep%20Purple/1998%20-%2030-%20Very%20Best%20Of/". In this case, the only files that are passed toPROPFINDare regular files, and so no ending slash is needed. And I've checked, noPROPFINDrequest returns any200 0response.My explanation is that
mpd, at some point inside the code atsrc/storage/plugins/CurlStorage.cxx, produces a wrongPROPFINDrequest on a directory with a missing trailing slash, this request produces a200 0response after a timeout (probably 60 seconds by looking at the timestamps). Becausempdcannot get any information on the directory, it removes it from the database. The files are then added again because this time the request is correctly built (notice the trailing slash) :I am not familiar with the code at
src/storage/plugins/CurlStorage.cxx, which is why I've come to bother you with this problem. I hope I have provided you with enough information. If not, please ask.I thank you in advance for your time, and I thank you again for this fantastic software.
Mit freundlichen Grüssen,
Aymeric Agon-Rambosson