Open labkode opened 2 years ago
@dragotin @micbar this is a good starting point. From this quick analysis I don't know if the explosion of the operations comes from the implementation of the filesystem (ocisfs) of from upper layers (gateway, storage provider, webdav, ...)
For eos most of the read xattrs are fetched with a single eos finfo
. If you take a look at the xattr names: tmtime
and treesize
aro covered by the eos finfo
. Measuring the number of xattr reads in decomposedfs and trying to inferring eos performance from it is wildly misleading, IMO.
Aggregating ListStorageSpace, Stat and GetPath into a single call seems to be the more fruitful approach: when making a PROPFIND /dav/space/{storageid$spaceid!opaqueid}/relative/path
request we can improve performance in two ways:
Name
and Path
properties to the CS3 ResourceInfo
and use a field mask to indicate which property is requested. (the list shares call will profit from this even more)Space
property to the CS3 ResourceInfo
so a space based Stat request can be routed based on the providerid and the space info can be returned if requested with a field mask.@butonic for xattr is understandable, however there a dozens of stat calls, which they will affect EOS.
- add dedicated
Name
andPath
properties to the CS3ResourceInfo
and use a field mask to indicate which property is requested. (the list shares call will profit from this even more)- add a
Space
property to the CS3ResourceInfo
so a space based Stat request can be routed based on the providerid and the space info can be returned if requested with a field mask.- use a field mask to make desktop client requests for the etag as cheap as possible. No need to build a full ResourceInfo, if all that is needed is the etag
CS3Api Change in https://github.com/cs3org/cs3apis/pull/173
Work in progress.
I suggest you do a profiling of the previous scenarios on edge to compare to baseline in master. The profiling will help to identify where the bottlenecks are, to remove the implementation of driver aside.
Work in progress.
This issue tracks IO patterns in the OCIS decomposed FS for optimisation. Testing setup: https://owncloud.dev/ocis/getting-started/ Username for testing: einstein/relativity Host machine: Centos 7 Version:
Version: \n Compiled: 2022-06-15 00:00:00 +0000 UTC
The
getxattr
systemcall fails very often, same examples:Empty personal space
As a user I click on the "All files tab" from the web, the following number of requests are made:
Creating a folder in the empty personal space
Listing the personal space that contains one folder
Listing the personal space containing 10 folders
Listing received shares panel (one share only)