micbar
commented
2 years ago Comments from ocis
Both CERN and ownCloud decided together the following scheme:
A resourceID contains either:
- I: only an 1) absolute path (Starting with a leading slash) and 2)no IDs
- II: or an 1) ID and 2) no path
- III: or an 1) ID and 2) a relative path (Always starting with a
.)
II was returning an absolute path in reva master. This is changed in the edge branch. A stat, like on the filesystem has no knowledge about the path, only the file basename. We know and return the parentID. The full path can be queried with GetPath().
Examples from @labkode
- A is a valid request and a valid response
- B and G this requests cannot work. If you don't send and ID, we cannot route the request. Clients with path based access need to call ListStorageSpaces with the path to lookup the spaceID.
Comments on the desired request/responses
Request: Response
res_id = nil res_id = {storage_id, opaque_id_c}
path = /top/a/b/c path = /top/a/b/c
parent_id = {storage_id, opaque_id_b}
# path without a res_id cannot work
eos file info /a/b/c
------------------------------------------------------------------------------------
Request: Response
res_id = {storage_id, opaque_id_c} res_id = {storage_id, opaque_id_c}
path = nil path = /top/a/b/c # edge is returning /c, needs to be fixed
parent_id = {storage_id, opaque_id_b}
# this request is valid, it behaves like a filesystem stat. `/c` is only the basename to the res_id. We cannot expect a path to the parent in case. To get the path, call `GetPath`
eos file info inode:opaque_id_c
------------------------------------------------------------------------------------
Request: Response
res_id = {storage_id, opaque_id_a} res_id = {storage_id, opaque_id_c}
path = ./b/c path = /top/a/b/c # edge is returning c, needs to be fixed
parent_id = {storage_id, opaque_id_b}
# same as above, c is the basename, we cannot expect that the path to the root is returned
path_a = eos file info inode:opaque_id_a
eos file info path_a/b/c
------------------------------------------------------------------------------------
Request: Response
res_id = {storage_id, opaque_id_c} res_id = {storage_id, opaque_id_c}
path = . path = /top/a/b/c # edge is returning c, needs to be fixed
parent_id = {storage_id, opaque_id_b}
# same as above
eos file info inode:opaque_id_c
------------------------------------------------------------------------------------
Request: Response # eosfs will always return HTTP 404 here
res_id = {storage_id, storage_id} res_id = {storage_id, storage_id}
path = nil path = nil # the "name" of the space root node is not exposed
parent_id = nil?
# path is not set when it is the root folder.
--
In the Request, nil or "." or "" paths SHALL be considered synonymsFirstly, examples A and G show a defective implementation that doesn't honour the CS3APIs. Paths are a first class citizen of the API and are expected to work.
We thought that we had a mutual agreement that the spaces are first class citizens and all paths are relative to a space.
For example, having the tree /a/b/c/d a client that works with paths can query /a, /a/b or/a/b/c.
A client that works with ids and that knows about d will need to query the parent of d, which is c, then the parent of c which is b and then the parent of b which is a. Performance will be a function of tree depth. A workaround you may suggest is to "remember" node information in the client, however this pushes complexity to any client using the API (desktop sync, mobile) and basically anyone willing to integrate with OCIS in an optimal way.
To illustrate this behaviour from UNIX/POSIX environment, we're inside folder /var/tmp/data/log/2022/05/03/proxy.
Let us elaborate on this.
You know the id of /var/tmp/data/log/2022/05/03/proxy
You do a stat() with the ID, the response will give you the parentID and the stat information.
If you need the path, you do a getPath() which will give you the root of the space and eg ./data/log/2022/05/03/proxy. So in this case /var/tmp/ is the space mountpoint.
Now, after two request you are able to do all kind of operations within the whole depth of this tree. We never encounter cases in where clients need to do the example
cd ..
cd ..
cd ..
cd ..
cd ..Summary
- Spaces are the new first class citizens
- Spaces always require an indirection and need to be discovered before clients can interact with files.
- The spaces indirection moves logic to the clients, that is explicitly desired and core of the spaces concept. The server should never assume a data layout on the client because the server has no knowledge of the client use case. E.g. a Desktop Client works always path-based because it creates a local POSIX tree. Web or mobile Clients on the contrary are not bound to these filesystem semantics and want can present spaces in more sophisticated ways.
- Spaces are not considered to come from one instance only. So spaceIDs are globally unique and clients can connect to federated storage spaces without changing the scheme.
@dragotin Please also add your thoughts, because you were always insisting on that level of indirection with the clients in mind.
dragotin
butonic
labkode
stale[bot]
This writeup comes from a deep investigation performed by @ishank011 @gmgigi96 @glpatcern as part of our activity to run OCIS edge branch at CERN.
Examples on edge branch running ocisfs
4c510ada-c86b-4815-8820-42cdf82c3d51-> / is the home space ID254d0b60-20e4-4340-9eae-6fa9103ae7d7-> /app-try7321538e-15da-4352-8dd7-d59b3319e7ef-> /app-try/app-new-try.txt"id": { "opaqueId": "4c510ada-c86b-4815-8820-42cdf82c3d51", "storageId": "4c510ada-c86b-4815-8820-42cdf82c3d51" }, "path": "/",
cs3.gateway.v1beta1.GatewayAPI@localhost:19000> call Stat