opencadc / storage-inventory

Storage Inventory components
GNU Affero General Public License v3.0
2 stars 9 forks source link

raven: support additonal operations (directions) #212

Open pdowler opened 4 years ago

pdowler commented 4 years ago

raven currently supports pullFromVoSpace (GET) and pushToVoSpace (PUT) clients need to be able to POST {metadata} to {storage URL} to update file metadata clients need to be able to DELETE {storage URL} to delete a file from the system

If a client negotiates a write, they get URLs to any site that allows the write (whether or not the artifact exists there) so the update or delete may subsequently fail. If the client negotiates a read, it will get URLs to sites with the artifact, but the wrong permission check will be done, the pre-auth URL(s) will not work for update or delete, and URL(s) to read-only sites will also be returned.

The VOSpace transfer model allows the direction to be one of four push|pull constants or a URI (for internal copy|move). We can define some special URI value to indicate update and/or delete so that raven knows to check write permission and only return URLs to existing copies. Options for direction value:

Either of these would work. No further extension would be required because this now covers all 4 (5 inc. HEAD) operations one might do at the files endpoint.

pdowler commented 2 years ago

Work on writing a robust client (cadc data) to put files revealed that when one negotiates a write they may also want to use that URL to read (HEAD at least) to verify the put... minoc currently has a workaround to allow HEAD with a WriteGrant (essentially treating it as a "read-write" grant. read-only and read-write are more consistent with permission concepts (node props) in VOSpace so maybe we should just be using "read-write" rather than "write" grants. This would effect cadc-permissions, baldur, raven, and minoc.

Still need a way to negotiate a read-write URL. Maybe raven could just prefer replacement of an existing file (pushToVoSpace) over writing to any available site? eg return URLs to all writable sites, but prioritize existing copies??