Closed nsymms closed 6 years ago
mar10
commented
6 years ago So I'm guessing this it's the client not setting "lastmodified" after the upload?
You could probably check that with high verbosity (-vv), however there may still a bug in verbose mode, that was fixed in the yet unreleased 2.4.1 version (and 3.0.0 pre).
nsymms
commented
6 years ago Thanks for the tip. I patched the verbose mode issues and got lots of useful info.
If I'm following the process correctly, here's what's happening when I drag a file from one Explorer window to the Explorer window with the wsgidav share (/beeline folder, for testing):
So the real question is why would the server responding with 403? Here's the first 403 response, in detail:
08:26:41.804 - <9764> wsgidav.debug_filter INFO : PROPPATCH Request ---
08:26:41.804 - <9764> wsgidav.debug_filter INFO : ACTUAL_SERVER_PROTOCOL: 'HTTP/1.1'
08:26:41.804 - <9764> wsgidav.debug_filter INFO : PATH_INFO : '/beeline/compat.py'
08:26:41.805 - <9764> wsgidav.debug_filter INFO : QUERY_STRING : ''
08:26:41.805 - <9764> wsgidav.debug_filter INFO : REMOTE_ADDR : '::1'
08:26:41.805 - <9764> wsgidav.debug_filter INFO : REMOTE_PORT : '61901'
08:26:41.805 - <9764> wsgidav.debug_filter INFO : REQUEST_METHOD : 'PROPPATCH'
08:26:41.805 - <9764> wsgidav.debug_filter INFO : REQUEST_URI : '/dav/beeline/compat.py'
08:26:41.805 - <9764> wsgidav.debug_filter INFO : SCRIPT_NAME : '/dav'
08:26:41.805 - <9764> wsgidav.debug_filter INFO : SERVER_NAME : 'WsgiDAV/2.4.0 Cheroot/6.3.1 Python/3.6.5'
08:26:41.806 - <9764> wsgidav.debug_filter INFO : SERVER_PROTOCOL : 'HTTP/1.1'
08:26:41.808 - <9764> wsgidav.debug_filter INFO : SERVER_SOFTWARE : 'WsgiDAV/2.4.0 Cheroot/6.3.1 Python/3.6.5 Server'
08:26:41.808 - <9764> wsgidav.debug_filter INFO : SERVER_PORT : '8080'
08:26:41.808 - <9764> wsgidav.debug_filter INFO : HTTP_CACHE_CONTROL : 'no-cache'
08:26:41.809 - <9764> wsgidav.debug_filter INFO : HTTP_CONNECTION : 'Keep-Alive'
08:26:41.809 - <9764> wsgidav.debug_filter INFO : HTTP_PRAGMA : 'no-cache'
08:26:41.809 - <9764> wsgidav.debug_filter INFO : HTTP_USER_AGENT : 'Microsoft-WebDAV-MiniRedir/10.0.16299'
08:26:41.809 - <9764> wsgidav.debug_filter INFO : HTTP_IF : '(<opaquelocktoken:0xc3bb9d725c7963eda26d3eb6795a3c63ce9cb0d52f8794d73fc4d7a81c22bce4>)'
08:26:41.809 - <9764> wsgidav.debug_filter INFO : HTTP_TRANSLATE : 'f'
08:26:41.809 - <9764> wsgidav.debug_filter INFO : HTTP_HOST : 'localhost:8080'
08:26:41.809 - <9764> wsgidav.debug_filter INFO : CONTENT_TYPE : 'text/xml; charset="utf-8"'
08:26:41.809 - <9764> wsgidav.debug_filter INFO : CONTENT_LENGTH : '443'
08:26:41.809 - <9764> wsgidav.debug_filter INFO :
08:26:41.810 - <9764> wsgidav.http_authenticator DEBUG : realm '/dav'
08:26:41.810 - <9764> wsgidav.http_authenticator DEBUG : No authorization required for realm '/dav'
08:26:41.810 - <9764> wsgidav.request_server DEBUG : RequestServer: __init__
08:26:41.811 - <9764> wsgidav DEBUG : parseIfHeaderDict
{'*': [[(True,
'locktoken',
'opaquelocktoken:0xc3bb9d725c7963eda26d3eb6795a3c63ce9cb0d52f8794d73fc4d7a81c22bce4')]]}
08:26:41.811 - <9764> wsgidav DEBUG : testIfHeaderDict(/dav/beeline/compat.py, ['opaquelocktoken:0xc3bb9d725c7963eda26d3eb6795a3c63ce9cb0d52f8794d73fc4d7a81c22bce4'], b5b8eb72de2fff55e2862d0fa672e108-1529069201-0)
08:26:41.811 - <9764> wsgidav.lock_manager DEBUG : checkWritePermission(/dav/beeline/compat.py, 0, ['opaquelocktoken:0xc3bb9d725c7963eda26d3eb6795a3c63ce9cb0d52f8794d73fc4d7a81c22bce4'], )
08:26:41.812 - <9764> wsgidav.lock_manager DEBUG : checking /dav/beeline/compat.py
08:26:41.812 - <9764> wsgidav.lock_manager DEBUG : l=Lock(<c3bb..>, '/dav/beeline/compat.py', , exclusive, depth-infinity, until 2018-06-15 14:26:41 (in 3599.986998796463 seconds)
08:26:41.812 - <9764> wsgidav.lock_manager DEBUG : checking /dav/beeline/
08:26:41.812 - <9764> wsgidav.lock_manager DEBUG : checking /dav/
08:26:41.812 - <9764> wsgidav.lock_manager DEBUG : checking /
08:26:41.812 - <9764> wsgidav INFO : PROPPATCH XML request body:
<?xml version='1.0' encoding='UTF-8'?>
<D:propertyupdate xmlns:D="DAV:" xmlns:Z="urn:schemas-microsoft-com:">
<D:set>
<D:prop>
<Z:Win32CreationTime>Thu, 14 Jun 2018 15:00:32 GMT</Z:Win32CreationTime>
<Z:Win32LastAccessTime>Fri, 15 Jun 2018 13:26:41 GMT</Z:Win32LastAccessTime>
<Z:Win32LastModifiedTime>Thu, 14 Jun 2018 15:00:32 GMT</Z:Win32LastModifiedTime>
<Z:Win32FileAttributes>00000000</Z:Win32FileAttributes>
</D:prop>
</D:set>
</D:propertyupdate>
08:26:41.813 - <9764> wsgidav.wsgidav_app INFO : ::1 - (anonymous) - [2018-06-15 13:26:41] "PROPPATCH /beeline/compat.py" length=443, depth=0, connection="Keep-Alive", agent="Microsoft-WebDAV-MiniRedir/10.0.16299", elap=0.009sec -> 207 Multi-Status
08:26:41.813 - <9764> wsgidav.debug_filter INFO : <9764> ---PROPPATCH Response(207 Multi-Status): ---
08:26:41.813 - <9764> wsgidav.debug_filter INFO : Content-Type: 'application/xml'
08:26:41.813 - <9764> wsgidav.debug_filter INFO : Date: 'Fri, 15 Jun 2018 13:26:41 GMT'
08:26:41.814 - <9764> wsgidav.debug_filter INFO : Content-Length: '643'
08:26:41.814 - <9764> wsgidav.debug_filter INFO :
08:26:41.814 - <9764> wsgidav.debug_filter INFO : PROPPATCH XML response body:
<?xml version='1.0' encoding='UTF-8'?>
<D:multistatus xmlns:D="DAV:">
<D:response xmlns:NS1="urn:schemas-microsoft-com:">
<D:href>/dav/beeline/compat.py</D:href>
<D:propstat>
<D:prop>
<NS1:Win32CreationTime/>
<NS1:Win32LastAccessTime/>
<NS1:Win32LastModifiedTime/>
<NS1:Win32FileAttributes/>
</D:prop>
<D:status>HTTP/1.1 403 Forbidden</D:status>
</D:propstat>
</D:response>
<D:responsedescription>403 Forbidden: Access denied to the specified resource
403 Forbidden: Access denied to the specified resource
403 Forbidden: Access denied to the specified resource
403 Forbidden: Access denied to the specified resource</D:responsedescription>
</D:multistatus>
08:26:41.814 - <9764> wsgidav.debug_filter INFO : <9764> --- End of PROPPATCH Response (643 bytes) ---To answer my own question:
The default file system provider has no propManager, so it can only handle generic DAV PROPPATCH properties. It doesn't recognize Microsoft's "Win32LastModifiedTime" et al. so it fails with 403.
So what next? Does someone need to write a Win32 fs_dav_provider?
mar10
commented
6 years ago Either this, or add optional support for {DAV}:Win32LastModifiedTime to the standard FilesystemProvider.
I would rather do this in a feature branch for 3.x however, since I am planning to feature-freeze 2.x.
Are you interested in working on this, or support with testing?
nsymms
commented
6 years ago I'll certainly help with testing anything Win32 related.
I could also do a pull request to add support to the standard FS provider, but the properties look like {urn:schemas-microsoft-com:}Win32LastModifiedTime. I'm not sure if you want to add those to the standard provider since they don't use {DAV:}.
I'd have to add support for the other 3 properties as well, since they're sent together. The request_server doPROPPATCH() is an all-or-nothing call, so handling one "set property" won't help unless all properties in the same request can be handled.
mar10
commented
6 years ago I think we can could add it to the standard, since M$ is a common-enough WebDAV client.
We should make it configurable, e.g. by an support_win32_properties option.
A PR for 3.x would be great.
nsymms
commented
6 years ago After spending some time researching this, here's what I've found:
So I think the solution might be a simple change to the documentation. Something like
N.B.: If you're going to use wsgidav to interact with the Windows file system, then you must use a persistent property manager, otherwise file attributes (create time, modified time, accessed time, file mode) will not translate to Windows.
My initial plan was to translate the MS extensions into standard WebDAV properties and therefore only provide partial support for them (without a prop manager). It seems like a hack, but I can probably still do that if there's interest.
Thoughts, anyone?
Edit: Ugh. Just discovered that, as stated above, Windows ignores the standard WebDAV server properties if it finds the Win32 ones. So if you modify a file on the share, it gets a new "getlastmodified" property value, but the old Win32 value doesn't change and that's the one Windows displays. :confounded: I guess I'm back to my original plan.
mar10
commented
6 years ago Thanks for digging in. So the minimal approach (without implementing the full MS extensions) would be to accept the Win32LastModifiedTime, ... properties on PROPPATCH requests and handle them by setting those times directly on the target files? Do we need to return this props an PROPFIND, or do the clients understand {DAV:}getlastmodified?
nsymms
commented
6 years ago Yes, the minimal approach is to simply translate a PROPPATCH of Win32LastModifiedTime into a PROPPATCH of {DAV:}getlastmodified. Code already exists to handle getlastmodified. The other 3 Win32 properties can be ignored, returning an OK response for them so that the whole PROPPATCH request succeeds.
I've tested this approach with Windows 10 and it seems to work well. For PROPFIND, the client simply uses getlastmodified in lieu of Win32LastModifiedTime.
There are some issues with Win7 that I'm looking at. It has an older WebDAV client with other problems.
nsymms
commented
6 years ago Just submitted PR #114. Sorry for the delay.
Tested with Windows 10 and works very well. With Windows 7 and it will never work. The Windows 7 WebDAV client is broken. It always tries to set the same values for all Win32 properties regardless if a file has been modified or accessed or not. I wasn't able to test with other Windows versions. My testing server is having issues. I can probably get others (Win8, 2008, 2012??) done this week. I suspect some will have similar issues as Win7. Microsoft always puts WebDAV on the back burner and only updates for security holes.
I had to make temp patches to other parts of the code in order to test v3.0.0a. I guess you're not quite finished refactoring. I left those untouched; if you want some pointers I can tell you or submit another PR.
mar10
commented
6 years ago Merged it, thanks!
I had to make temp patches to other parts of the code in order to test v3.0.0a. I guess you're not quite finished refactoring. I left those untouched; if you want some pointers I can tell you or submit another PR.
Yes I this was a lot of renaming and PyCharm refactoring probably did not all references right. That's why I consider it alpha for now. If you could send your changes as PR that would certainly help.
I can't find any other comments on this, so I have to assume it's unique to my setup or it's by design and I missed it in the documentation somewhere.
I'm using Windows 10 with Python 3.6.5, wsgidav 2.4.0. When I run
wsgidav --host=0.0.0.0 --port=8080 --root=c:\users\my_user:I can connect to the share without problems, view and download files without problems. But when I upload a file to the webdav share (I use drag-drop), it gets a new timestamp of "now". Strangely the new file initially shows its original timestamp, but a refresh (F5) will show the new "now" timestamp.
Is this by design? It's hard for me to believe that it is because it makes any sort of file sync impossible.
NB: I also see this same behavior in SeaFile, which uses the wsgidav library.
Edit: I also tried using a config file and setting
mutable_live_props = ["{DAV:}getlastmodified"]which didn't help. So I'm guessing this it's the client not setting "lastmodified" after the upload?