Until now we've been sort of cheating with etags. When you request the etag of a file we hash its location, mtime, and size and call it a day. This generally works, but isn't recommended.
Owncloud requests etags for directories, as well. When it asks for the etag of a top-level directory, we can't afford to recurse all the way through its subdirectories looking for changes. So we've started saving hidden .jsdav files in each directory that contain the etags of the directory's files and subdirectories. Computing a file's etag involves md5ing its contents; computing a directory's etag means md5ing the directory's .jsdav file, and thereby hashing its files' hashes.
PROPFIND performance was pretty bad when the directory got large (> 1000 files) but with some in-memory etag caching it went from 35s to 3s for PROPFIND on a 5000-file directory.
Until now we've been sort of cheating with etags. When you request the etag of a file we hash its location, mtime, and size and call it a day. This generally works, but isn't recommended.
Owncloud requests etags for directories, as well. When it asks for the etag of a top-level directory, we can't afford to recurse all the way through its subdirectories looking for changes. So we've started saving hidden
.jsdavfiles in each directory that contain the etags of the directory's files and subdirectories. Computing a file's etag involvesmd5ing its contents; computing a directory's etag means md5ing the directory's.jsdavfile, and thereby hashing its files' hashes.PROPFINDperformance was pretty bad when the directory got large (> 1000 files) but with some in-memory etag caching it went from 35s to 3s forPROPFINDon a 5000-file directory.